[RELEASE] IMDB Ratings for Movies

cosmicr · 2011-01-26 13:39

Scidd0w Wrote:cosmicr;

Thank you very much for this addon! I'm really looking forward to using it.
I'm on windows7 though!

give it a try - it may work.

about SQL, well I'm not very good at SQL, I copied all the queries from the php code. I'm not sure about using mysql instead of sqlite. I suppose the script would first have to check what the database type is. Or are the two types compatible? I'll look into it, but it may be beyond my capabilities.

Targettio · 2011-01-26 15:30

cosmicr Wrote:give it a try - it may work.

I am at work atm, so can't test but I had a quick look over the code and I don't see any major reasons why the plugin wouldn't work on windows/mac (although could have missed something).

It may not find the file paths correctly, but I think that's about it. I will give it a shot tonight.

Scidd0w · 2011-01-26 16:59

I'm also at work atm.
I'll give it a spin when I get home!

atconc · 2011-01-27 00:03

I started messing around with the code to see if I can get a stand-alone version working that connects to the myqsl database instead of using the sqllite stuff - it's been a while since I've done any programming and I've never used python before so it's somewhat slow going!

I was hoping that I could just swap the sqllite apis for mysql ones and everything else would just work but it seems that the database schema's are not the same so I have more to do. I'll try to keep chipping away at it in spare moments (or until someone more competent steps in).

So far I have a Python connecting to the database but get the below error:

Code:

Traceback (most recent call last):

  File "imdb.py", line 85, in <module>

    c.execute('SELECT c09, c00, idMovie FROM movie WHERE idMovie > ?',(lastid,))

  File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 151, in execute

    query = query % db.literal(args)

TypeError: not all arguments converted during string formatting

So looks like I will need to work on the database stuff a bit

Scidd0w · 2011-01-27 00:52

Just tested it and it installs and works under windows7 as well.
I do get an error after a while though. See below fo a copy of the log.

Code:

23:48:30 T:3460 M:1448738816  NOTICE: IMDB Rating Script: Opening http://www.imdb.com/title/tt0211915

23:48:31 T:3460 M:1446731776  NOTICE: IMDB Rating Script: Le Fabuleux Destine d'Amlie Poulain(119) rating: 8.6/10

23:48:31 T:3460 M:1446727680   ERROR: Error Type: exceptions.UnicodeEncodeError

23:48:31 T:3460 M:1446727680   ERROR: Error Contents: 'ascii' codec can't encode character u'\xe9' in position 55: ordinal not in range(128)

23:48:31 T:3460 M:1446727680   ERROR: Traceback (most recent call last):

                                              File "C:\Users\zwartewaal\AppData\Roaming\XBMC\addons\script.imdbratings\default.py", line 114, in ?

                                                xbmc.executebuiltin('xbmc.Notification(IMDB Ratings,'+title[j]+')')

                                            UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 55: ordinal not in range(128)

atconc · 2011-01-27 01:04

got a bit further with making it work with mysql

Code:

c.execute('SELECT c09, c00, idMovie FROM movie WHERE idMovie > ?',(lastid,))

needs to be changed to

Code:

c.execute('SELECT c09, c00, idMovie FROM movie WHERE idMovie > [b]%s[/b]',(lastid,))

(mysqldb uses %s instead of ? as a placeholder)

Now it runs through all the movies in the db ok but fails when it tries to scrape the first movie with

Code:

Traceback (most recent call last):

  File "imdb.py", line 118, in <module>

    string =  d_notify + title[j] + "(" + str(lid[j]) + ")" + " rating: " + str(rating[0]) + "/10"

IndexError: list index out of range

time for bed so will look at this another time

cosmicr · 2011-01-27 01:14

atconc Wrote:

Code:

Traceback (most recent call last): File "imdb.py", line 118, in <module> string = d_notify + title[j] + "(" + str(lid[j]) + ")" + " rating: " + str(rating[0]) + "/10" IndexError: list index out of range

time for bed so will look at this another time

that line's not a show stopper, take it out and see if it works. if it does send me the code and I'll integrate it into the addon.

I suppose the user will have to specify their database type (with the default being sqlite). Unless someone knows how to check the type first?

cosmicr · 2011-01-27 01:19

Scidd0w Wrote:Just tested it and it installs and works under windows7 as well.
I do get an error after a while though. See below fo a copy of the log.

Code:

23:48:30 T:3460 M:1448738816 NOTICE: IMDB Rating Script: Opening http://www.imdb.com/title/tt0211915 23:48:31 T:3460 M:1446731776 NOTICE: IMDB Rating Script: Le Fabuleux Destine d'Amlie Poulain(119) rating: 8.6/10 23:48:31 T:3460 M:1446727680 ERROR: Error Type: exceptions.UnicodeEncodeError 23:48:31 T:3460 M:1446727680 ERROR: Error Contents: 'ascii' codec can't encode character u'\xe9' in position 55: ordinal not in range(128) 23:48:31 T:3460 M:1446727680 ERROR: Traceback (most recent call last): File "C:\Users\zwartewaal\AppData\Roaming\XBMC\addons\script.imdbratings\default.py", line 114, in ? xbmc.executebuiltin('xbmc.Notification(IMDB Ratings,'+title[j]+')') UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 55: ordinal not in range(128)

oops, I missed that line, it was the same problem as earlier - non-ascii (utf-8) symbols seem to stuff it up - which most french titles seem to have Smile

I will fix it later.

atconc · 2011-01-27 13:33

cosmicr Wrote:that line's not a show stopper, take it out and see if it works. if it does send me the code and I'll integrate it into the addon.

I suppose the user will have to specify their database type (with the default being sqlite). Unless someone knows how to check the type first?

I think that line builds the IMDB url so is pretty critical? If I comment it out then the run fails because the variable string is undefined. Will keep playing with it...

edit - ignore me - it's just building a like to print in the log

atconc · 2011-01-27 14:08

atconc Wrote:edit - ignore me - it's just building a like to print in the log

It seems like I've got the database part working but the scraping isn't working for me

the str(rating[0]) part is what's throwing the error about the array index being out of bounds which suggests that the rating is never being fetched (I think?)

I put a few print statements in and it looks like I never get into the

if(rating)
if(votes) or
if(top250)

branches of the code so it now just runs through listing out the url for each of the movies like this:

Code:

IMDB Rating Script: 279 Movies to update

IMDB Rating Script: Opening http://www.imdb.com/title/tt0088846

IMDB Rating Script: Opening http://www.imdb.com/title/tt0063350

IMDB Rating Script: Opening http://www.imdb.com/title/tt0248808

IMDB Rating Script: Opening http://www.imdb.com/title/tt0381849

I've not really got any idea how to debug the scraper so I think I'm stuck for now.

Targettio Senior Member Posts: 160 Joined: Aug 2010 Reputation: 0	2011-01-26 15:30 Post: #12 cosmicr Wrote:give it a try - it may work. I am at work atm, so can't test but I had a quick look over the code and I don't see any major reasons why the plugin wouldn't work on windows/mac (although could have missed something). It may not find the file paths correctly, but I think that's about it. I will give it a shot tonight.
	find quote

Scidd0w Junior Member Posts: 25 Joined: Dec 2010 Reputation: 0	2011-01-26 16:59 Post: #13 I'm also at work atm. I'll give it a spin when I get home!
	find quote