2008-08-11, 12:02
nobody who cares has stepped up
2008-08-11, 12:45
Thanks for the info spiff. Guess I will give it a try, Im by no means an expert at xml but it shouldnt be that difficult.
2008-08-11, 13:22
xml isnt the "difficult" part, but rather the regular expressions.
shoot any specific question my way i gladly answer them
shoot any specific question my way i gladly answer them
2008-08-12, 12:48
Is this currently being developed? For some reason one of the TV scanned Witchbade (so one anime folder out of 110 ;()..so just wonder what's happening.
Tar
Tar
2008-08-12, 16:19
Many tv and movies databases includes animes, its not related.
The regexp are indeed tedious to get done but I think I've got the hang of it
I started a scraper based on anidb.net but had to switch for animenfo.com for now because a search result from anidb.net is answered as gzip encoded. Is there any way to get past that
I should have the animenfo one useable by the end of the week.
The regexp are indeed tedious to get done but I think I've got the hang of it
I started a scraper based on anidb.net but had to switch for animenfo.com for now because a search result from anidb.net is answered as gzip encoded. Is there any way to get past that
I should have the animenfo one useable by the end of the week.
2008-08-12, 17:13
i'll gladly add gzip support for you - we already support zip. however it will have to be after the feature freeze though
2008-08-12, 20:11
Sure, no worries I have enough material for the time being to work at least for a animenfo scraper 
2008-08-12, 23:33
Remember to checkout the XBMC online manual:
http://wiki.xbmc.org/?title=Category:Scraper
...and feel free to add anything to the manual that you think is missing (it is a wiki after all)
http://wiki.xbmc.org/?title=Category:Scraper
...and feel free to add anything to the manual that you think is missing (it is a wiki after all)
2008-12-27, 18:06
this topic started years ago. Hope someone can update us on the status of the project
2009-01-14, 23:36
I'd be pretty interested in this feature as well ... even if it's only for having a nice thumb for the series, and an item in the library for every episode. That way I can remember where I was in a certain series.
2009-01-26, 11:42
Right ... I jump in the topic, and I hope my post will fit ...
So I had nearly my whole week-end coding the scraper for anidb ... and the proof of concept says ... it work perfectly ...
BUT ... and that's why i don't post it yet ...
My scraper is an early beta which has several problems :
1)The problem of gzipped page at anidb ... well I made a dirty hack : a perl script on my webserver is called with the anidb you want as a parameter ... the script only performs a "wget <url> |gunzip" ... dirty but it works like a charm ...however if i was to release the scraper everyone would use my IP to access anidb ... and there comes the real problem ...
2)AniDB, does not allow scraping, because it is seen as crawling/mirrorring their website ... and considering all the work it took to build such a database, I understand. So the scraper works, but you get banned after too many requests, updating your anime series.
I would have loved to release this scraper, because it seems some people wanted it, and because I was proud to see it work that nicely ...
The only valid way of interrogating AniDB, would be via their API, which is not web based, but UDP based ... I have to see what would the cost for me to code and UDP-to-XBMC interface ...
Anyone got their hands on the UDP API ?
So I had nearly my whole week-end coding the scraper for anidb ... and the proof of concept says ... it work perfectly ...
BUT ... and that's why i don't post it yet ...
My scraper is an early beta which has several problems :
1)The problem of gzipped page at anidb ... well I made a dirty hack : a perl script on my webserver is called with the anidb you want as a parameter ... the script only performs a "wget <url> |gunzip" ... dirty but it works like a charm ...however if i was to release the scraper everyone would use my IP to access anidb ... and there comes the real problem ...
2)AniDB, does not allow scraping, because it is seen as crawling/mirrorring their website ... and considering all the work it took to build such a database, I understand. So the scraper works, but you get banned after too many requests, updating your anime series.
I would have loved to release this scraper, because it seems some people wanted it, and because I was proud to see it work that nicely ...
The only valid way of interrogating AniDB, would be via their API, which is not web based, but UDP based ... I have to see what would the cost for me to code and UDP-to-XBMC interface ...
Anyone got their hands on the UDP API ?
2009-01-26, 12:04
hey,
interest again. i will whip up a solution for 1) asap
interest again. i will whip up a solution for 1) asap
2009-01-26, 12:59
Great ... however, the gzip problem isn't really the current issue 
It will take time to developp the UDP interface .... Maybe it could be done via a python script ... but can a python plugin be used as a scraper ?
Anyway writing the scraper was really interesting ... and seeing it working (before I got banned twice by anidb) was really beautiful :p
PS : If I have some time I should go on and edit the scraper wiki page to add the GetEpisode, etc functions ...
It will take time to developp the UDP interface .... Maybe it could be done via a python script ... but can a python plugin be used as a scraper ?
Anyway writing the scraper was really interesting ... and seeing it working (before I got banned twice by anidb) was really beautiful :p
PS : If I have some time I should go on and edit the scraper wiki page to add the GetEpisode, etc functions ...
