Developing an Amazon Movie Scraper

jelockwood,

I am looking forward to testing your scraper out. I have been following this thread since Oct. I have found many of the DVD's on IMDB don't have coverart, where as Amazon does, as well as missing dvd information. I appreciate all your efforts, and I am looking forward to testing this out.

gyrene2083 Wrote:jelockwood,

I am looking forward to testing your scraper out. I have been following this thread since Oct. I have found many of the DVD's on IMDB don't have coverart, where as Amazon does, as well as missing dvd information. I appreciate all your efforts, and I am looking forward to testing this out.

If the only problem is cover art, then you could use the IMDB scraper, and manually select a local picture, or put a picture in the directory with a .tbn file extension. I wrote the scrapers because some titles are not listed at all on IMDB and I still wanted to include them in the XBMC library.

The download link is now live so you can give it a go.

both are now sitting in svn (r16563). cheers again!

I just tried using the Amazon scrapers I mostly wrote for the first time for several weeks, and damn they don't work any more for me.

Currently neither is finding any results (so it is not simply an issue of scraping info from a selected result). This was the original problem that I had (constructing a correct query in the scraper, and then getting/showing the list of results). This was originally solved by C-Quel generously providing his original Amazon scraper effort which I then finished off.

Could anyone else confirm whether the Amazon scrapers (either US or UK) are currently still working for them, and if so what DVD title they used successfully.

If on the other hand, other users confirm it is broken, would anyone be able to assist in diagnosing it?

What held me up last time, is that I could not (without a LAN packet sniffer) see what request the scraper sent out, and what result it got back from Amazon and then be able to see how far it got. Once I got past this and moved on to scraping the film info, this could be easily tested by seeing how many fields successfully returned results.

Try this...

change Get SearchResults from

imageColumn"[^:]*a href="([^"]*)"[^:]*[^>]*alt="([^"]*)"

productTitle"><a href="([^"]*)"> ([^<]*)</a>

or properly formatted

productTitle&quot;&gt;&lt;a href=&quot;([^&quot;]*)&quot;&gt; ([^&lt;]*)&lt;/a&gt;

might not be perfect as i simply glanced at amazon no tools to hand.

I have already thanked C-Quel (again) via a private message, but this fix does so far look successful. I will do some more testing and then put updated versions on my download page and issue a request for them to be included as updated and fixed versions in XBMC.

Many thanks again to C-Quel and everyone else who has helped out in the past.

C-Quel Wrote:Try this...

change Get SearchResults from

imageColumn"[^:]*a href="([^"]*)"[^:]*[^>]*alt="([^"]*)"

productTitle"><a href="([^"]*)"> ([^<]*)</a>

or properly formatted

productTitle&quot;&gt;&lt;a href=&quot;([^&quot;]*)&quot;&gt; ([^&lt;]*)&lt;/a&gt;

might not be perfect as i simply glanced at amazon no tools to hand.

jelockwood Wrote:I have already thanked C-Quel (again) via a private message, but this fix does so far look successful. I will do some more testing and then put updated versions on my download page and issue a request for them to be included as updated and fixed versions in XBMC.

Many thanks again to C-Quel and everyone else who has helped out in the past.

Please use our tracker instead and attach a unified diff to the previous scraper.

Amazon does not give permission to get info via http. They have a webservice to use which is legal, however you have to delete the info after 3 months... hehe this means that movies should automaticly start to disappear from the library if they were scanned via Amazon webservice scrapper

both scrapers disabled in svn

ultrabrutal Wrote:Amazon does not give permission to get info via http. They have a webservice to use which is legal, however you have to delete the info after 3 months... hehe this means that movies should automaticly start to disappear from the library if they were scanned via Amazon webservice scrapper

you gonna ruin everything