+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305)
+----- Thread: [WIP] AniDB.net Anime Video Scraper (/showthread.php?tid=64587)
- bambi73 - 2011-06-14
I updated my collection of Anime studio logos. Feel free to use them, naming should match with results of AniDB.net scraper.
How to use them? It depends on your skin, in Alaska Revisited they should be placed in addons/skin.alaska.revisited/media/flags/studios directory. In other skins it should be similar, if you are not sure please ask in your skin subforum/thread, i can't help you with it. Please be aware that not all skins support studio logos/flags.
awesome work - lad1337 - 2011-06-15
first i wanted to say thank god someone is working on this ! thank you
im am currently developing the anime support for sickbeard
and i am planing to integrate full anidb support (in some ways it already has)
thing is i use xbmc and always need to make sure that the files/file names i create are compatible with xbmc
so far i relied on thetvdb for show information (i figure you all know what the problems are...)
now i saw your anime-list.xml and look at it and nearly died !!
this is absolutely awesome ! i was planing on some sort of andb:tvdb map
but could not imagen it to this extend !
my main question is would you allow me to use this map in myAnime branch ?
it would be part of an opensource program licensed under gpl
or could i refere to this xml in my code ?
you can get basic information about sickbeard itself here: http://sickbeard.com/
information about my anime branch here: http://www.sickbeard.com/forums/viewtopic.php?f=4&t=1452
and all code is on github! main and myAnime
and did you think about publishing this scraper on github ?
it would be easier to maintain and easier to get (i dont know if i found the newest version in this thread :/ )
this would also improve the managing of other contributing to the scraper or anime-list
hosting the anidb.xml and anime-list.xml would be obsolete too
we do use a similar system for name exceptions
lad1337
ps i now nothing about developing a scraper for xbmc but is it possible to add a python script to it ? if so might want to look at adba
- bambi73 - 2011-06-16
Feel free to use anime-list.xml, only please mention me somewhere, I spent A LOT of time on it
Scraper itself is hosted on Google code, but best is use one published in official XBMC addon repo because in svn can be some unstable changes.
As you already know anidb.xml and anime-list.xml are hosted on Google sites. I never thought about hosting it on Google code too, but maybe it's not so bad idea becuase i guess there are more possibilities how to grant access for other maintaners. Till now you are one of few who are interested in anime-list.xml so there never was demand for any sharing :p.
About XBMC scrapers, they doesn't support Python. But anyway i'll check you code because i plan to make some AniDB.net support script for XBMC someday in future, so some sample code for libpyanidb will be helpful.
- lad1337 - 2011-06-16
bambi73 Wrote:Feel free to use anime-list.xml, only please mention me somewhere, I spent A LOT of time on itoff course ... credit where credit is due!
bambi73 Wrote:Scraper itself is hosted on Google code, but best is use one published in official XBMC addon repo because in svn can be some unstable changes.ohh i didnt notice it in the repo thats awesome .. will look there
bambi73 Wrote:As you already know anidb.xml and anime-list.xml are hosted on Google sites. I never thought about hosting it on Google code too, but maybe it's not so bad idea becuase i guess there are more possibilities how to grant access for other maintaners. Till now you are one of few who are interested in anime-list.xml so there never was demand for any sharing :p.i am not sure if i wil link directly to your file (hosted on google) or maintain a mirror
although that would be a only changing a url in the code
do you know how many ppl use you scraper ? to get an idea how tested the anime-list.xml is
bambi73 Wrote:About XBMC scrapers, they doesn't support Python. But anyway i'll check you code because i plan to make some AniDB.net support script for XBMC someday in future, so some sample code for libpyanidb will be helpful.mmhh how can i put it ... i wouldnt use libpyanidb any more specially because some comands dont work any more since the api has changed
adba is based on libpyanidb and i am developing it at the moment... i would gladly help you with any questions
although in some/one point(s) libpyanidb is superior e.g. local caching
fyi here is some sample code
while looking at anime-list.xml one question came up.
what is anidbseason="0" or anidbseason="1" ?
lad
- bambi73 - 2011-06-16
lad1337 Wrote:off course ... credit where credit is due!I think it's up to you, direct link will provide you always actual state.
ohh i didnt notice it in the repo thats awesome .. will look there
i am not sure if i wil link directly to your file (hosted on google) or maintain a mirror
although that would be a only changing a url in the code
lad1337 Wrote:do you know how many ppl use you scraper ? to get an idea how tested the anime-list.xml isI have no idea how much people use it, there are some stats in official addon XBMC repo (28353 downloads), but it means count of downloads/updates and say nothing about how much people really use it for scraping.
How is it tested? It depends how you mean it. Mapping on anime/show level is tested quite good because it contain almost all my anime and i tested scraped info one by one. Mapping on episode level is bit worse because it's used only for loading episode plot/picture so you will not find problems until you really watch it and checking what is written there.
By default episode numbering is same on both sites, when multiple seasons are present on TheTVDB then mapping is used, but it's generated by program so it should be ok (unless i screwed episode count for each season input to my program). Unfortunatelly there are a lot of irregularities in numbering, sometimes i noticed it but i guess i missed a lot
So it's far from 100%.lad1337 Wrote:mmhh how can i put it ... i wouldnt use libpyanidb any more specially because some comands dont work any more since the api has changedThat XBMC script is still only idea so i don't know about current state, i only notices this library year ago when i was thinking about how to make AniDB.net scraper. I'll check your updates when i start
adba is based on libpyanidb and i am developing it at the moment... i would gladly help you with any questions
although in some/one point(s) libpyanidb is superior e.g. local caching
fyi here is some sample code
lad1337 Wrote:while looking at anime-list.xml one question came up.anidbseason="0" are specials, anidbseason="1" are regular episodes
what is anidbseason="0" or anidbseason="1" ?
lad
thetvdb.org and xbmc recognize specials as season 0, so this is same for anidb.net mapping.
- zephyr001 - 2011-06-29
Yesterday I tried to add some new episodes to my anime library but the Anidb scraper stopped picking them up even though they were named the same way as the files already in the library.
I tried rebuilding the library from scratch. The scraper would download TV show information, but the episodes are not being added. The same thing happened after a complete reinstall of xbmc.
I saw Finalspace's post from 6/1 that said he had similar issues, but I can see in cache\scrapers\metadata.anidb.net that I'm getting show information back so I'm not banned.
I've tested this on Windows 7 x64 and Ubuntu 11.04 server running xbmc-live, using the latest stable builds and anidb 1.2.0. Same results on both machines.
Here's the log from Windows 7 machine running xbmc http://pastebin.com/6TSxaMqS.
Tested with a single show: Ano Hi Mita Hana... From the log file it looks like XBMC is finding the shows but the scraper is unable to download episode information.
- zephyr001 - 2011-06-29
Oh forgot to mention, I would appreciate any help to figure out why this is happening. I'm stumped, spent a good hour or two trying to narrow down the source of the problem.
- bambi73 - 2011-06-29
I was wondering how long it'll take till someone notice. Five days is quite good, looks like someone is really using this scraper
Right now scraper is broken on episode level. Guys from AniDB.net added voice actor/seiyuu picture information to xml returned by http API but looks like they added/changed it bit more. These additional changes broke scraper.
I already corrected it localy but want to finish some other tweaks before I release it. It's almost done, so I expect there will v1.3.0 release tomorrow evening.
- zephyr001 - 2011-06-30
Oh and I thought I broke something since no one else seemed to be having trouble. Thanks!
Blank episode list - timoteoandre - 2011-07-01
Please help me out here guys
I had to reinstall xbmc, and now my anime list wont show the episodes.
The scrapper finds and list the animes, but the number of episodes is always 0.
I had it working before, but now I can't remember how this advancedsettings.xml was configured.
I've already tried using the one in page 21, or the new one in page26, even the extra regex in one of the threads, but it nevers shows up.
My animes are usually [Fansub]ANIME - epXX.ext
Though one in particular "FLCL" is set as "FLCL 00X.ext"
I think my xbmc is not even using the advancedsettings, I placed on this folder:
C:\Users\User\AppData\Roaming\XBMC\userdata\advancedsettings.xml
And its currently like this:
Code:
<advancedsettings>
<tvshowmatching action="prepend">
<regexp>ep([0-9]+)</regexp>
</tvshowmatching>
</advancedsettings>Gosh, it was working so nice before...
EDIT:
Ohh, I read every freaking page on the internet trying to get a solution, and I missed the most obvious post right on top of mine.
EPIC FAIL
Guess I will just wait for the update.
- ZERO <ibis> - 2011-07-02
I am out of town for a few days (normally update every few days with new anime) and the scrapper is brokeded
Hopefully there will be an update soon, thanks for the hard work!
- musera - 2011-07-04
I'm starting to really feel the lack of this scraper. Without a doubt it is the most important part of xbmc for me.
Hopefully you get the update out soon.
- bambi73 - 2011-07-04
1.3.0:
Added: Actors thumbnail
Added: Store AniDB.net ID in database (maybe it'll be useful in future
)Changed: Episodes list correction (unexpected change in xml returned from AniDB.net)
Changed: 1->2 episode mapping correction
Changed: Small plot filtering improvement
Should be available soon from both XBMC repos
- salival - 2011-07-05
I think it now queries anidb too much, because the scraper gets temporarily banned and puts loads of xml files in the cache folder (270, about the number of anime series I have on the share). This is with a regular scan for new files, not for a complete rescan.
- bambi73 - 2011-07-06
Unfortunatelly, it has IMHO nothing to do with scraper (additionally nothing related to AniDB.net data loading changed), it's controled by XBMC if directory is scraped for changes. XBMC stores hash keys for each path/directory in DB and when this hash change then it means it needs to update/scrape it. Few weeks ago XBMC cleared all hashes in DB (don't ask me why, really have no idea
) and results was same like yours, it scraped every single anime in my library and i got temp-banned. To solve this problem i modifed scraper related code to not allow more than one request per 10sec, but it's only my local change so it doesn't help you so much. From my point of view it'll be best to scrape your anime from scratch, right now i have no better idea (or you can do it one by one manualy).