Regular updates of IMDb rating (also with other scrapers)

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
XBL. Offline
Senior Member
Posts: 140
Joined: Aug 2006
Reputation: 1
Thumbs Up  Regular updates of IMDb rating (also with other scrapers)
Post: #1
Themoviedb is great for fetching movie info (and default for many, these days), but the rating is for most movies not really accurate. IMDb has better ratings (at least from way more people), so getting the info from themoviedb and the rating from IMDb would be great. Also a periodically update of the ratings would be nice.

I wrote a small PHP script that will exactly do this. It's not very nice, but it does the job.

Features:
- Update ratings using IMDb, keep up to date with new movies.
- Update the whole database to get new ratings.


It works on XBMCLive, most probably any Ubuntu box with XBMC. I guess other Linux distro's work as well, but that's untested.

The readme included contains all the info on how to use it. I will try to help people, but I'm busy with other stuff as well so don't count on me ;-).

UPDATE May 4th 2010:

ezechiel1917 made a couple of changes to the code which resulted in imdbratings 1.2. All credits go to him for this version.

Upgrade instructions are in the new readme.

CHANGELOG:

1.2:
- Cosmetic changes to the code.
- Number of votes are updated in DB.
- If movie is in the IMDb top250, this is added to the DB.

Version 1.2: http://dump.troep.net/public/imdbratings_1.2.zip
Version 1.0: http://dump.troep.net/public/imdbratings.zip
(This post was last modified: 2010-05-04 13:25 by XBL..)
find quote
ezechiel1917 Offline
Aeon Group
Posts: 732
Joined: Jun 2005
Location: Czech Republic
Post: #2
Thanks! Love it. Worked with quite large library without a glitch. Nod

Updating IMDb ratings for all movies is really handy, but it would be great and more data consistent to also update user votes of new ratings (maybe even top250 ranks). What do you think? Wink

XBMCbuntu 13.0 | XBMC PRE-14.0 | VDR + VNSI @ Technotrend CT-3650 CI | Transparency! skin | trakt.tv
find quote
XBL. Offline
Senior Member
Posts: 140
Joined: Aug 2006
Reputation: 1
Post: #3
It would be better, yes. Unfortunately I did not write the class accessing the IMDb site and as it does not implement all the features, I can not implement it.

The best way to go would be to create a new scraper from a combination of the themoviedb scraper and the imdb scraper, but I have no time and not enough knowledge to do this. Maybe this will catch on and get someone working on a new scraper, but for now this is the most we can get.
find quote
ezechiel1917 Offline
Aeon Group
Posts: 732
Joined: Jun 2005
Location: Czech Republic
Post: #4
would you be able to update your script if we could find out how to retreive votes and top250 rank?

could this be enough to add it to class.media.php?

Code:
$arr['votes'] = $this->match('/(([0-9]{1,3}[,])*[0-9]{1,3}) votes/ms', $html, 1);
$arr['top250'] = $this->match('/Top 250: #([0-9]{1,3})/ms', $html, 1);

XBMCbuntu 13.0 | XBMC PRE-14.0 | VDR + VNSI @ Technotrend CT-3650 CI | Transparency! skin | trakt.tv
find quote
XBL. Offline
Senior Member
Posts: 140
Joined: Aug 2006
Reputation: 1
Post: #5
If the regexes work, it will probably be enough. When I get the time, I'll implement it. Hopefully somewhere in the coming days. Thanks for the code Smile.

Just to be clear: I did not write the class, but got it from phpclasses.org. I'll update the other script to represent the GPL the class is released under (my source is public domain).
find quote
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #6
I'm at work right now, I just saw this. How does this work? Does it update the database, or does it generate an NFO. Would you care to make a NFO version which can be run as a user job from within mythicalLibrarian or provide the update methods so that I can recode this in bash to be used in mythicalLibrarian?

mythicalLibrarians rating system is broken for me because schedules direct does not provide rating information, but they provide a Zap2it id, title, subtitle, original airdate and other episode information. I'd love to be able to obtain rating information for those shows which do not have it on thetvdb and themoviedb, so during NFO generation, or even after, mythicalLibrarian could call up imdb and grab that if it comes out with no information.

I will be checking this out tonight.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find quote
XBL. Offline
Senior Member
Posts: 140
Joined: Aug 2006
Reputation: 1
Post: #7
It does a direct update on your database (in a quite ugly way, since there is no .deb available for php5-sqlite3 in the current XBMCLive - I call the sqlite3 binary directly with exec() ).

I'm not planning to change the script to do something else then update the database directly (so no nfo generation). For me it's just a quick piece of code to get better ratings with my movies in XBMC, and my idea is not to expand it too much.

If someone else is willing to make a version with more features, be my guest to reuse my code.
find quote
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #8
I will read your PHP tonight and try to come up with a way to grab information from imdb. It's just a matter of figuring out how to obtain the ratings from the website and how to identify the name of the show.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find quote
ezechiel1917 Offline
Aeon Group
Posts: 732
Joined: Jun 2005
Location: Czech Republic
Post: #9
Anything new regarding updates for votes and imdb top250 rank?

XBMCbuntu 13.0 | XBMC PRE-14.0 | VDR + VNSI @ Technotrend CT-3650 CI | Transparency! skin | trakt.tv
find quote
XBL. Offline
Senior Member
Posts: 140
Joined: Aug 2006
Reputation: 1
Post: #10
I've just updated the first post to include the new version (1.2). Ezechiel1917 added some lines of code to update both the number of votes and the imdb top250 ranking. Thank ezechiel1917 for this Smile.

You can download the new version here: http://dump.troep.net/public/imdbratings_1.2.zip
find quote
hebjor Offline
Junior Member
Posts: 11
Joined: Jan 2009
Reputation: 0
Post: #11
Thanks mate, this is great stuff!

Currently updating my database now, works like a charm Smile
find quote
JustinAiken Offline
Fan
Posts: 668
Joined: Feb 2007
Reputation: 0
Location: XBMC+Unraid=Happy!
Post: #12
Does this work with the current Dharma RC and the new IMDB changes?

XBMC: Frodo 12.2, with Transparency!
17.5 TB Unraid Server: 2500 movies, 8800 episodes of TV Apps: SABnzbd, Sickbeard, Couchpotato, mySQL
HTPC: Win8 (cause Steam), i7 920, ATI 6850 OC
Watching on: Panasonic TC65-PS64 with lowend Sony 5.1 HTIB
find quote
roror Offline
Junior Member
Posts: 12
Joined: Mar 2010
Reputation: 0
Post: #13
does it work on windows machines?
find quote
gmk2 Offline
Member
Posts: 86
Joined: Jan 2010
Reputation: 0
Post: #14
A quick look suggests this only supports the old sqlite db, so won't work with mysql database.
find quote
Jefman Offline
Junior Member
Posts: 18
Joined: Nov 2009
Reputation: 0
Location: Gothenburg
Post: #15
gmk2 Wrote:A quick look suggests this only supports the old sqlite db, so won't work with mysql database.

Hi,

Anyone feels to update this script to work with new database ? I would like it to have Imdb rating instead.

Thanks !
find quote
Post Reply