themoviedb scraper now partly broken

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
edz2k9 Offline
Senior Member
Posts: 159
Joined: Dec 2009
Reputation: 0
Exclamation  themoviedb scraper now partly broken
Post: #1
Hi all

running on the last stable linux release am I right in thinking a change to the themoviedb api has now broken fan art thumbnails and other information.

Recent scrapes have picked up some details, notably title, year and some info but thumbnails and fan art dont appear even though they are on the site (I've added them in some cases).

Checking the log shows lines such as

DEBUG: scraper: GetTMDBFanart returned <details><fanart></fanart></details>

so it would appear that although the initial query to themoviedb has brought back lots of xml nodes of

<poster id="4bc919a1017a3c57fe009d8d">
<image url="http://i1.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-original.jpg" width="1000" size="original" height="1500"/>
<image url="http://i2.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-mid.jpg" width="500" size="mid" height="750"/>
<image url="http://i1.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-cover.jpg" width="185" size="cover" height="278"/>
<image url="http://i3.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-thumb.jpg" width="92" size="thumb" height="138"/>
</poster>


and

<backdrop id="4bc9199e017a3c57fe009d6d">
<image url="http://i2.themoviedb.org/backdrops/d6d/4bc9199e017a3c57fe009d6d/beetle-juice-original.jpg" width="1920" size="original" height="1080"/>
<image url="http://i2.themoviedb.org/backdrops/d6d/4bc9199e017a3c57fe009d6d/beetle-juice-poster.jpg" width="780" size="poster" height="439"/>
<image url="http://i1.themoviedb.org/backdrops/d6d/4bc9199e017a3c57fe009d6d/beetle-juice-thumb.jpg" width="300" size="thumb" height="169"/>
</backdrop>

They aren't being parsed properly anymore. I took a look at share/bin/xbmc/scripts/scrapers/video/tmdb.xml but although it looks like a regexp nodelist I couldnt seem to fix anything in there.

Any ideas, this is a pain in the rear when it goes wrong especially since I'm still trying to add a backlog of dvd's over into the system

Thanks

Ed
find quote
HioMrSan Offline
Junior Member
Posts: 2
Joined: May 2010
Reputation: 0
Post: #2
Wondering the same. Using the live rc1. I just tried to rebuild my movie library and I'm getting weird results. It's not pulling any fanart or posters, although its getting synopsis and cast. A handful of movies work correctly, and all the others exhibit the problematic behavior.
find quote
theuni Offline
Team-XBMC Communication Manager
Posts: 1,105
Joined: Oct 2007
Reputation: 2
Location: Atlanta, Ga, USA
Post: #3
debug logs please.
find quote
aptalca Offline
The Dude
Posts: 916
Joined: Sep 2009
Reputation: 25
Post: #4
here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly
(This post was last modified: 2010-05-01 06:31 by aptalca.)
find quote
Trozza Offline
Junior Member
Posts: 8
Joined: May 2010
Reputation: 0
Post: #5
Thanks! Working now, was having the same issue as the guys above but only just noticed it about 10 minutes ago when I tried to add a new movie. Thanks again.
find quote
odin836 Offline
Junior Member
Posts: 8
Joined: Feb 2008
Reputation: 0
Post: #6
Still not 100%, is it possible not all areas have been updated?

*Prior to modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...

Scraper not receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

*After modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

Scraper not receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...
find quote
HioMrSan Offline
Junior Member
Posts: 2
Joined: May 2010
Reputation: 0
Post: #7
aptalca Wrote:here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly

Thanks, this made some progress. Still not fully working. Some still experience the same problem, others work great. I'll have to grab a debug later.
find quote
AaronD Offline
Senior Member
Posts: 252
Joined: Jan 2007
Reputation: 0
Location: Dubai, United Arab Emirates
Post: #8
The problem is TMDB changed their API so that now the ID field for images is a GUID, not a simple number. The current REGEX is matching ID on numeric digits only, but now it can contain the alphas a-f as well. I am looking for at fixing this - I'll post a new Xml file if no one else gets to it first. Shouldn't be a difficult fix.
find quote
AaronD Offline
Senior Member
Posts: 252
Joined: Jan 2007
Reputation: 0
Location: Dubai, United Arab Emirates
Post: #9
Can someone try the following xml?

http://pastebin.com/DTCa46NN

Again, replace your common TMDB.XML only, not the main one. Let me know if it works for you.
find quote
BugBoy Offline
Junior Member
Posts: 29
Joined: Jan 2010
Reputation: 0
Post: #10
The latest XML doesn't fix the problem on my system. The file is also a lot bigger than the original file. Is that correct?
find quote
odin836 Offline
Junior Member
Posts: 8
Joined: Feb 2008
Reputation: 0
Post: #11
AaronD Wrote:Can someone try the following xml?

http://pastebin.com/DTCa46NN

Again, replace your common TMDB.XML only, not the main one. Let me know if it works for you.

Thanks for the try AaronD, but it does not resolve the issue on my system...
Missng i'd say one third of the FanArt and Thumbs for my library.
Sad
find quote
traeblain Offline
Junior Member
Posts: 8
Joined: Mar 2010
Reputation: 0
Post: #12
aptalca Wrote:here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly

I'm not having the same issues as everyone else. This version has updated 100% of my movies (~260 that I've transfered so far). So just an FYI. Using 9.11 live install.
find quote
Second Tier Offline
Junior Member
Posts: 8
Joined: Apr 2010
Reputation: 0
Post: #13
aptalca Wrote:here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly

Thanks aptalca and AaronD for the quick work on this!!

The quoted above fixed the problem for me, but only for the four most recent movies I've added. Don't want to blow away my entire library just yet.

thumbsup!
find quote
aptalca Offline
The Dude
Posts: 916
Joined: Sep 2009
Reputation: 25
Post: #14
odin836 Wrote:Still not 100%, is it possible not all areas have been updated?

*Prior to modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...

Scraper not receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

*After modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

Scraper not receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...

I just checked the format the api delivered for "16 blocks", "50 first dates" and "Aeon Flux" and they all conform to the new style. tmdb's database might have had been partially updated before you tried to scrape, but looks like they should work now.

FYI. I didn't rescrape my entire library to test it, I just rescraped the latest few movies in my library.
find quote
Cheeshead Offline
Junior Member
Posts: 17
Joined: Nov 2009
Reputation: 0
Post: #15
Thanks so much! Was pulling my hair out all night...updated XML file worked like a charm!
find quote
Post Reply