Does XBMC and XBMC Media Center end-users have permission to scrape imdb.com website?

ultrabrutal · 2009-01-12 10:12

Just to take one, imdb:

IMDB Conditions

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below."

Do XBMC have this permission? Just curious...

**spiff** · 2009-01-12 13:37

no, i was not aware of those terms (or atleast i could claim so).

not so much any longer. r17032 disables the scraper

ultrabrutal · 2009-01-12 14:47

Shit! What a party pooper I turned out be Sad

I'm afraid that most sites have conditions like this so most scrapers are in fact violating terms of the site

**theuni** · 2009-01-12 15:59

Quote:We do allow the limited use of robots and crawlers, such as those from certain search engines, with our express written consent. If you are interested in receiving our express written permission to use robots or crawlers on our site, please contact our Licensing Department.

I'd be happy to kick-off communications. I'd think this is the type of project that they would be open to.

Spiff: Let me know if this is something I should look into, or if it's up to the XBMC team to handle.

TheUni

**spiff** · 2009-01-12 16:01

i think we will have to handle this ourself.

thanks for the offer though

ultrabrutal · 2009-01-12 16:19

I'm afraid we eventually have to create our own movie database ala the mymovies database which we can all contribute to Sad

**spiff** · 2009-01-12 16:20

themoviedb.org is just that. i have already written the scraper a few months ago, but as the site had so little info i didn't bother with polishing it.

**pike** · 2009-01-12 18:21

Let me quote Conan O Brien "Nicely Done" Cool

ultrabrutal Wrote:Just to take one, imdb:

IMDB Conditions

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below."

Do XBMC have this permission? Just curious...

ultrabrutal · 2009-01-12 18:32

Pike, when commitmonitor showed me the commits spiff had done I felt really bad. I use the scrapers also hehe. Of course XBMC cannot violate any licenses, so I totally get spiff.

spiff, I didn't know about themoviedb.org. Do you use an api or really scrape the html? I don't see an imdb link, so you use the movie name as a param?

**spiff** · 2009-01-12 18:41

it's an xml based api.

see r17035

spiff Grumpy Bastard Developer Posts: 12,178 Joined: Nov 2003 Reputation: 82	2009-01-12 13:37 Post: #2 no, i was not aware of those terms (or atleast i could claim so). not so much any longer. r17032 disables the scraper Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.
	find

ultrabrutal Posting Freak Posts: 954 Joined: Feb 2005 Reputation: 0 Location: South of Heaven	2009-01-12 14:47 Post: #3 Shit! What a party pooper I turned out be I'm afraid that most sites have conditions like this so most scrapers are in fact violating terms of the site
	find

spiff Grumpy Bastard Developer Posts: 12,178 Joined: Nov 2003 Reputation: 82	2009-01-12 16:01 Post: #5 i think we will have to handle this ourself. thanks for the offer though Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.
	find

ultrabrutal Posting Freak Posts: 954 Joined: Feb 2005 Reputation: 0 Location: South of Heaven	2009-01-12 16:19 Post: #6 I'm afraid we eventually have to create our own movie database ala the mymovies database which we can all contribute to
	find

ultrabrutal Posting Freak Posts: 954 Joined: Feb 2005 Reputation: 0 Location: South of Heaven	2009-01-12 18:32 Post: #9 Pike, when commitmonitor showed me the commits spiff had done I felt really bad. I use the scrapers also hehe. Of course XBMC cannot violate any licenses, so I totally get spiff. spiff, I didn't know about themoviedb.org. Do you use an api or really scrape the html? I don't see an imdb link, so you use the movie name as a param?
	find

spiff Grumpy Bastard Developer Posts: 12,178 Joined: Nov 2003 Reputation: 82	2009-01-12 18:41 Post: #10 it's an xml based api. see r17035 Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.
	find