[WIP] AniDB.net Anime Video Scraper

Hi!

I am running into trouble when adding new anime episodes recently. This happens when adding a new series as well when adding episodes from a series where the adding of episodes previously worked. Here is some log information when trying to refresh an episode that was earlier added successfully but now no longer is:



I do not know if this is enough or if additional information is needed. It seems the episode is identified but somehow no related information is found. When using the TVDB scraper the series seems to be identified correctly.

Never mind, it is working again now.

(2012-03-15 13:40)shinner Wrote:  Having trouble scraping episode info for some newer anime though (eg. Another). I pinpointed the problem to anime-list.xml, it's not mapped in there. Am I right that this file has not been updated for a while, like since late 2011?
So I'll have to use my own personal anime mapping? (It also works if I edit the xml in the cache, but then it'll get overwritten )

I was completelly busy with my work and real life for last 4-5 months so all my XBMC related projects was stoped, i was using it only for occasional watching . I hope it'll be better in following months, so I plan to update anime-list.xml with newest Animes (btw if you already created your own personal list please send it to me by PM, i'll use it as base for my updates), but can't promise when it'll be done .
Anyway I'm not sure if anime-list.xml is source of your problems. It's only mapping between anidb.net and thetvdb.com, so it affect only loading informations from thetvdb.com, like background images or episode plots, rest informations from anidb.net should be there.

(2012-03-15 13:40)shinner Wrote:  Edit:
If anyone has used personal anime mapping before, I could use a little help. If I enable the XBMC web server, can I place the pers-anime-list.xml locally? If so, where do I have to place it and what's the url? (http://localhost. something) And do I have to disable the default xml file?

When I added this feature to scraper I tested it with XBMC web server and it was working fine, but don't ask me details, it was 1,5y ago

i just wanted to let you guys know that i am working on a project with a few others to takle this problem once and for all

i am talking about http://thexem.de/
it is ment to map every episode on one entity with the appropriate one on any other entity

for now there is only a proxy service for the tvdb api but i am personally interested to do the same for anidb
what the proxy service does: you give a origin(e.g. tvdb) and a destination(e.g. anidb) then use the api logic of the origin get a response just like the origin would give you but with the numbering of the destination...
we are still in a closed testing phase but its looking good for a tvdb->scene relation already

but you can already get every map with a json api

what i could do is implement a auto generated anime-list.xml based on the info provided in xem
my question is would this be something of use or would you rather prefer something else to improve the scraper
or would you accept/wish another xml structure or json? could you even use a non local xml for the anime-list.xml ?
if not how could xem provide a anidb_id<->tvdb_id mapping in a way that this scraper would benefit ?

lad1337

(2012-03-29 12:49)bambi73 Wrote:  I hope it'll be better in following months, so I plan to update anime-list.xml with newest Animes (btw if you already created your own personal list please send it to me by PM, i'll use it as base for my updates), but can't promise when it'll be done .
Anyway I'm not sure if anime-list.xml is source of your problems. It's only mapping between anidb.net and thetvdb.com, so it affect only loading informations from thetvdb.com, like background images or episode plots, rest informations from anidb.net should be there.

My bad, I should've been clearer in explaining. Yup, the series, actors etc. scrape ok, it's just the episode plots are empty. I only just added one anime in manually to test the issue, so no point sending it to you, sorry ^^' Thanks for your work!
Can I ask how you generated your anime-list.xml? Did you write everything manually, or there's a way to automate it by parsing the files returned by tvdb and anidb API with a script? Sorry if I don't make sense, I don't really get the hang of how the scraper works yet, so help is really appreciated.

(2012-03-29 12:49)bambi73 Wrote:  

(2012-03-15 13:40)shinner Wrote:  Edit:
If anyone has used personal anime mapping before, I could use a little help. If I enable the XBMC web server, can I place the pers-anime-list.xml locally? If so, where do I have to place it and what's the url? (http://localhost. something) And do I have to disable the default xml file?

When I added this feature to scraper I tested it with XBMC web server and it was working fine, but don't ask me details, it was 1,5y ago

I'll have to do some trial and error when I'm free then. I'll update you with the settings when I can get it to work XP

@lad1337 Not a programmer, so can't really answer your questions, but that sounds really promising in simplifying episode mapping.

@shinner: I have modified version of scraper which outputs results (in XML form) into debug log. But putting it togerther is manual work.
@lad1337: I'll check your site again when it's populated with data and will see if it can be used in scraper in some way. Anyway it can't be full-fledged replacement for anime-list.xml because it contains some additional informations or corrections for existing data.

Quote:I'll explain the basics, I really am short on time to solve the problem totally >.<. Do you know how to read XML files? The file in question is
C:\Users\<your user name>\AppData\Roaming\XBMC\cache\scrapers\metadata.anidb.net\anime-list.xml

Basically what the anidb scraper does is it gets some data from Anidb, then it maps the episode to TVDB and gets episode plot info from there. The mapping is done in anime-list.xml, eg.
<anime anidbid="6239" tvdbid="88651" defaulttvdbseason="1">
maps anidb ID 6239 to TVDB ID 88651. It's the same show (07-Ghost in this case).

If you know how to, you can copy one of the animes, modify the name, anidbid and tvdbid (get those from the sites yourself). Then scrape. It's more complicated for multi-season anime, so I'm not going to talk about it for now.

Do a backup of your edited file, because the cache will be cleared after a while. The personal anime mapping setting of the addon is supposed to work-around this, but I can't get it working yet

hi can someone please help me understand what i have to do here? i dont really know what i must do. Im trying to scrape episode info for shows like one piece, naruto shippuuden, another... any help, would be much appreciated

(2012-03-13 17:32)shinner Wrote:  Your anime names are originally absolute numbering right? eg. Naruto 305, not S15E02 or something? If so, try to use regex, it's much less trouble.
Now, create an advancedsettings.xml file, and add this in:

Code:

<advancedsettings>
    <tvshowmatching action = "prepend">
        <regexp>Anime[\\/].*[/\._ \-]()([0-9]+)(-[0-9]+)?</regexp>
     </tvshowmatching>
</advancedsettings>

Edit the "Anime" in the code above to the name of your source folder. This regex should handle multi-episode files.

Thanks to gates' post.

Ok, I am trying to get my AniDB scraper working, and I have down the above for my advanced settings.

I have also named half my anime using the anidb own file renamer script.

An example of how my files are named now is : Code Geass Hangyaku no Lelouch - 04 - His Name is Zero - [gg](49715425)

An Example of how My files used to be named ( and work with TVDB) is : K-ON!.S01E08.Freshman Reception!

Now with the new Regex, Code Geass for example with the new episode name setup does not show any episodes at all, all my anime wit the old setup ( S00e00) show up the episodes but with about 10 of the same episode ove rand over.

Why is this not working?

Also another problem I am having is it is refusing to use Banners, no matter how many times I tell it to in the settings. And TVDB does have the banners as before I changed to AniDB they all had banners perfectly.

I have noticed the banners options keeps deactivating in anidb options and I have to keep re-enabling it. It just does not want to stay on or get any banners...

Only anime it has got a banner for is Code Geass, which is also the one anime I am trying the new name setup on and is showing 0 episodes.

I finally got it working by using Regex of

<regexp> - ([0-9]+)</regexp>

However anidb plugin even with banners enabled in settings still REFUSES to download banners, and I am left with Large Posters SQUASHED into banner size.

Really driving me nuts now..

@Reser: When you enable banners it doesn't mean they will be default automatically. If banner exist for anime it's only added as another possibility, so you can choose it from Info window. Problem with banners is that they don't exists for each anime, so I did it working this way. Additionally default poster is always from anidb.net and make default something from thetvdb.com, like posters, will make scraper handling even more complicated than it's now.