[DEPRECATED]Rotten Tomatoes ratings scraper

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
mortstar Offline
Senior Member
Posts: 282
Joined: Aug 2010
Reputation: 3
Post: #46
FreakinAye Wrote:Boo. Such an awesome scraper but all my media has NFOs (new ones with just an imdb link) and is stored in a shared mysql db. it doesn't look like there is any way this can work for me?

The scraper uses the same Nfo retrieval as the Team XBMC IMDb scraper. If you want, you can set the scraper as IMDb + RT on your movies source and refresh the info on one of your movies. I imagine it will work.

EDIT: Just updated to 1.3.7 to match Team XBMC's scraper changes i.e. "fixed: missing imdb ID in search results due to site changes"

[Image: all-banner.jpg]
(This post was last modified: 2011-07-31 23:05 by mortstar.)
find quote
freakshow33 Offline
Junior Member
Posts: 3
Joined: Aug 2011
Reputation: 0
Post: #47
Mortstar, this addon sounds awesome -- however, I can't get it to work for some odd reason. IMDB+RT installs and scrapes fine, for everything BUT the RT ratings. The logs even seem to show it grab everything fine from RT except for the ratings:

---
22:37:20 T:3572 DEBUG: ------ Window Init (DialogProgress.xml) ------
22:37:21 T:3572 DEBUG: CVideoDatabase::GetMovieId (smb://XBMC/Videos/Temp/Movies/About Schmidt/About Schmidt.avi), query = select idMovie from movie where idFile=55
22:37:21 T:3572 DEBUG: CAnnouncementManager - Announcement: OnRemove from xbmc
22:37:21 T:3572 DEBUG: GOT ANNOUNCEMENT, type: 8, from xbmc, message OnRemove
22:37:21 T:2652 DEBUG: msg: PICTURE::LoadImage: Unable to open image: special://masterprofile/Thumbnails/Video/Fanart/b5817e47.tbn Error: (2)
22:37:21 T:2652 ERROR: Texture manager unable to load file: special://masterprofile/Thumbnails/Video/Fanart/b5817e47.tbn
22:37:21 T:2652 DEBUG: msg: PICTURE::LoadImage: Unable to open image: special://masterprofile/Thumbnails/Video/b/b5817e47.tbn Error: (2)
22:37:21 T:2652 ERROR: Texture manager unable to load file: special://masterprofile/Thumbnails/Video/b/b5817e47.tbn
22:37:23 T:3572 DEBUG: CVideoDatabase::GetMovieId (smb://XBMC/Videos/Temp/Movies/About Schmidt/About Schmidt.avi), query = select idMovie from movie where idFile=55
22:37:23 T:3572 DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'smb://XBMC/Videos/Temp/Movies/About Schmidt/About Schmidt.avi'
22:37:23 T:3024 DEBUG: Thread CVideoInfoDownloader start, auto delete: 0
22:37:23 T:3024 DEBUG: ADDON::CScraper::GetVideoDetails: Reading movie 'http://akas.imdb.com/title/tt0257360/' using IMDb + RT scraper (file: 'C:\Users\Administrator\AppData\Roaming\XBMC\addons\metadata.imdbrt.com', content: 'movies', version: '1.3.9')
22:37:23 T:3024 DEBUG: FileCurl::Open(0014B190) http://akas.imdb.com/title/tt0257360/
22:37:23 T:3024 DEBUG: scraper: GetDetails returned <details><id>tt0257360</id><originaltitle>About Schmidt</originaltitle><chain function="GetIMDBAKATitlesById">tt0257360</chain><year>2002</year><mpaa>Rated R for some language and brief nudity</mpaa><chain function="GetIMDBCert">tt0257360</chain><tagline>Schmidt Happens</tagline><runtime>125</runtime><chain function="GetRTRatingById">tt0257360</chain><genre>Comedy</genre><genre>Drama</genre><country>USA</country><studio>New Line Cinema</studio><outline>A man upon retirement embarks on a journey to his estranged daughter&#x27;s wedding only to discover more about himself and life than he ever expected.</outline><plot>A man upon retirement embarks on a journey to his estranged daughter&#x27;s wedding only to discover more about himself and life than he ever expected.</plot><chain function="GetRTOutlineById">tt0257360</chain><chain function="GetIMDBPlotById">tt0257360</chain><chain function="GetIMDBScaledCastById">tt0257360</chain><chain function="GetIMDBDirectorsById">tt0257360</chain><chain function="GetIMDBWritersById">tt0257360</chain><chain function="GetTMDBThumbsByIdChain">tt0257360</chain><chain function="GetTMDBFanartByIdChain">tt0257360</chain><chain function="GetHDTrailersnet480p">About Schmidt</chain><chain function="GetIMDBThumbsById">tt0257360</chain></details>
22:37:23 T:3024 DEBUG: scraper: GetIMDBAKATitlesById returned <details><url cache="tt0257360-combined.html" function="ParseIMDBAKATitles">http://akas.imdb.com/title/tt0257360/combined</url></details>
22:37:23 T:3024 DEBUG: FileCurl::Open(0014B190) http://akas.imdb.com/title/tt0257360/combined
22:37:24 T:3024 DEBUG: scraper: ParseIMDBAKATitles returned <details><title>About Schmidt</title></details>
22:37:24 T:3024 ERROR: ADDON::CScraper::Run: Unable to parse web site
22:37:24 T:3024 DEBUG: scraper: GetRTRatingById returned <details><url function="ParseRTTomAll" cache="0257360-rt.html">http://www.rottentomatoes.com/alias?type=imdbid&s=0257360</url></details>
22:37:24 T:3024 DEBUG: FileCurl::Open(0014B190) http://www.rottentomatoes.com/alias?type...&s=0257360
22:37:26 T:3024 ERROR: ADDON::CScraper::Run: Unable to parse web site
22:37:26 T:3024 DEBUG: scraper: GetRTOutlineById returned <details><url function="ParseRTOutline" cache="0257360-rt.html">http://www.rottentomatoes.com/alias?type=imdbid&s=0257360</url></details>
22:37:26 T:3024 DEBUG: scraper: ParseRTOutline returned <details><outline>In this funny, touching character study, Nicholson gives one of the best performances of his career.</outline></details>
---

It seems to throw "Unable to parse web site" when trying to grab RT ratings. Any clue what might be going on?

Platform is Windows, XBMC is the latest nightly build as of 3 nights ago, using default Confluence skin.

If there's some way to debug this offline with you, let me know -- it would be appreciated!
find quote
freakshow33 Offline
Junior Member
Posts: 3
Joined: Aug 2011
Reputation: 0
Post: #48
I don't think this is unique to me -- I tried another computer with a fresh install of the last official stable release and see the same problem.

However, it ONLY fails if set to Tomatometer + All Critics.

These work fine: Tomato + Top, Average + All, Average + Top.
find quote
freakshow33 Offline
Junior Member
Posts: 3
Joined: Aug 2011
Reputation: 0
Post: #49
Figured it out.

In rt.xml, the two lines that have this:

---
<expression noclean="1">v:average&quot;&gt;.*?([0-9]+).*?count&quot;&gt;([0-9]+)</expression>
---

Need to change to:

---
<expression noclean="1">&quot;all-critics-numbers&quot;.*?([0-9]+).*?Count&quot;&gt;([0-9]+)</expression>
---

Everything works fine now!
find quote
mortstar Offline
Senior Member
Posts: 282
Joined: Aug 2010
Reputation: 3
Post: #50
Thanks for the heads-up freakshow, looks like RottenTomatoes have changed their website and so broken the ratings scraping. I've not had a lot of free time since your post to look in to this, but I'll try and get a fix done today.

EDIT: Just seen your fix, that'll save me plenty of time - good man Smile

EDIT 2: Common scraper functions are now grabbing the proper values again.

[Image: all-banner.jpg]
(This post was last modified: 2011-08-27 12:49 by mortstar.)
find quote
Pete7874 Offline
Member
Posts: 66
Joined: Jun 2009
Reputation: 0
Post: #51
This is a nice plugin. Would it be possible to have the "Audience" rating option, too? Personally, I value the RottenTomato's audience ratings more than the critics' ratings.

XBMC 11.0 (Eden) on W7 Home Premium 64bit, Pentium Dual-Core E5300, ATI Radeon HD 4350 512MB; Samsung UN55B7100; Mediagate MCE remote (AVS GP-IR01BK)
find quote
mortstar Offline
Senior Member
Posts: 282
Joined: Aug 2010
Reputation: 3
Post: #52
Pete7874 Wrote:This is a nice plugin. Would it be possible to have the "Audience" rating option, too? Personally, I value the RottenTomato's audience ratings more than the critics' ratings.

Hi Pete

I did contemplate this, but the option to use IMDb ratings is within the scraper. Isn't this the same data as what you are asking but with a bigger sample?

[Image: all-banner.jpg]
find quote
Pete7874 Offline
Member
Posts: 66
Joined: Jun 2009
Reputation: 0
Post: #53
mortstar Wrote:Hi Pete

I did contemplate this, but the option to use IMDb ratings is within the scraper. Isn't this the same data as what you are asking but with a bigger sample?
Hmm... depending on the particular movie, there is often a sizable difference between the two. Take "MacGruber" for example. IMDB gave it 5.4 while RT audience gave it 3.4 (34%).

I've grown accustomed to deciding what I want to watch based on RT audience ratings. But maybe I'm the only one.

It'd be really cool if the scraper could pull and display both RT critics and RT audience ratings side by side, but now I'm probably just pushing it. Big Grin

XBMC 11.0 (Eden) on W7 Home Premium 64bit, Pentium Dual-Core E5300, ATI Radeon HD 4350 512MB; Samsung UN55B7100; Mediagate MCE remote (AVS GP-IR01BK)
(This post was last modified: 2011-09-29 17:51 by Pete7874.)
find quote
ronlut Offline
Junior Member
Posts: 4
Joined: Oct 2011
Reputation: 0
Post: #54
Pete7874 Wrote:Hmm... depending on the particular movie, there is often a sizable difference between the two. Take "MacGruber" for example. IMDB gave it 5.4 while RT audience gave it 3.4 (34%).

I've grown accustomed to deciding what I want to watch based on RT audience ratings. But maybe I'm the only one.

It'd be really cool if the scraper could pull and display both RT critics and RT audience ratings side by side, but now I'm probably just pushing it. Big Grin

Agree with that.
Would like to see that feature too (or at least the option to choose the audience ratings as the primary)
find quote
mortstar Offline
Senior Member
Posts: 282
Joined: Aug 2010
Reputation: 3
Post: #55
Been a while coming (I've had an extremely busy month In-Real-Life) but I've eventually got around to updating the addon with the Team XBMC changes to the IMDb scraper. Meaning now there is the ability to link directly to a YouTube trailer.

I'll look into adding Audience rating scraping from the RottenTomatoes website when I get the time, though I'm not wholly convinced that it offers anymore meaningful information than the IMDb user score (General audience rating with a greater sample size = more representative score)

[Image: all-banner.jpg]
(This post was last modified: 2011-11-25 20:26 by mortstar.)
find quote
macf1an Offline
Senior Member
Posts: 224
Joined: Aug 2011
Reputation: 1
Post: #56
moments ago - right after youtube was pushed to 2.8.0 this got marked as broken..
maybe there's no connection but anyway

ATV2 4.3 8F455 Seas0npassed / XBMC 11
Samsung HT-P70 5.1 / Samsung WS-32Z419P
(This post was last modified: 2011-12-07 00:40 by macf1an.)
find quote
mortstar Offline
Senior Member
Posts: 282
Joined: Aug 2010
Reputation: 3
Post: #57
Over the weekend I updated with Team XMBC's scraper changes as IMDb changed their website and genre, country, director and writer weren't scraping properly. They should all be working now.

macf1an Wrote:moments ago - right after youtube was pushed to 2.8.0 this got marked as broken..
maybe there's no connection but anyway

Do you still get this issue? Are you sure you are using IMDb+RT?

[Image: all-banner.jpg]
find quote
andyh69uk Offline
Junior Member
Posts: 11
Joined: Mar 2011
Reputation: 0
Post: #58
Hi,

First off, awesome scraper, thanks for all the work. I have set the options to scrape UK ratings, if I log in to the database it is grabbing UK rating however, I am using Aeon Nox and it requires them to be stored in the database as UKTongueG (for example). If I manually change the entry to include the 'UK:' then it works, without doing this it displays 'NR'. My questions, one is this a known problem and two is there a work around (other than making local changes to the add-on code)

Andy
find quote
sourbob Offline
Member
Posts: 69
Joined: Jun 2011
Reputation: 0
Post: #59
If I understand correctly, as of Eden, the basic IMDb scraper automatically does movie sets. Is that on the horizon for this scraper as well?

I hope so. I'd love to have that feature, but I find the RT scraping so fantastically preferable, that there's no way I'd ever switch back.
find quote
mortstar Offline
Senior Member
Posts: 282
Joined: Aug 2010
Reputation: 3
Post: #60
andyh69uk Wrote:Hi,

First off, awesome scraper, thanks for all the work. I have set the options to scrape UK ratings, if I log in to the database it is grabbing UK rating however, I am using Aeon Nox and it requires them to be stored in the database as UKTongueG (for example). If I manually change the entry to include the 'UK:' then it works, without doing this it displays 'NR'. My questions, one is this a known problem and two is there a work around (other than making local changes to the add-on code)

Andy

I'll see what I can rustle up. Personally I just want the UK rating without anything else as the skin I use just displays the text value. As this is set by skin-authors on an adhoc basis (and there really isn't a consensus that I know about) everybody wants something different.

I'm thinking an option where you can specify a string to prepend to the rating data scrapped from IMDb would be the best solution.

sourbob Wrote:If I understand correctly, as of Eden, the basic IMDb scraper automatically does movie sets. Is that on the horizon for this scraper as well?

I hope so. I'd love to have that feature, but I find the RT scraping so fantastically preferable, that there's no way I'd ever switch back.

I'm pretty sure this is only available in TheMovieDB scraper...I'll have a look if I can get this data out.

[Image: all-banner.jpg]
find quote
Post Reply