[WIP] AniDB.net Anime Video Scraper

  Thread Rating:
  • 3 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
bambi73 Offline
Senior Member
Posts: 217
Joined: Jan 2010
Reputation: 0
Location: Czech Republic
Post: #406
@shinner: I have modified version of scraper which outputs results (in XML form) into debug log. But putting it togerther is manual work.
@lad1337: I'll check your site again when it's populated with data and will see if it can be used in scraper in some way. Anyway it can't be full-fledged replacement for anime-list.xml because it contains some additional informations or corrections for existing data.
find quote
wentzelfcp Offline
Junior Member
Posts: 15
Joined: Mar 2012
Reputation: 0
Post: #407
Quote:I'll explain the basics, I really am short on time to solve the problem totally >.<. Do you know how to read XML files? The file in question is
C:\Users\<your user name>\AppData\Roaming\XBMC\cache\scrapers\metadata.anidb.net\anime-list.xml

Basically what the anidb scraper does is it gets some data from Anidb, then it maps the episode to TVDB and gets episode plot info from there. The mapping is done in anime-list.xml, eg.
<anime anidbid="6239" tvdbid="88651" defaulttvdbseason="1">
maps anidb ID 6239 to TVDB ID 88651. It's the same show (07-Ghost in this case).

If you know how to, you can copy one of the animes, modify the name, anidbid and tvdbid (get those from the sites yourself). Then scrape. It's more complicated for multi-season anime, so I'm not going to talk about it for now.

Do a backup of your edited file, because the cache will be cleared after a while. The personal anime mapping setting of the addon is supposed to work-around this, but I can't get it working yet

hi can someone please help me understand what i have to do here? i dont really know what i must doFrown. Im trying to scrape episode info for shows like one piece, naruto shippuuden, another... any help, would be much appreciated Big Grin
find quote
Reser Offline
Junior Member
Posts: 5
Joined: Sep 2011
Reputation: 0
Post: #408
(2012-03-13 17:32)shinner Wrote:  Your anime names are originally absolute numbering right? eg. Naruto 305, not S15E02 or something? If so, try to use regex, it's much less trouble.
Now, create an advancedsettings.xml file, and add this in:

Code:
<advancedsettings>
    <tvshowmatching action = "prepend">
        <regexp>Anime[\\/].*[/\._ \-]()([0-9]+)(-[0-9]+)?</regexp>
     </tvshowmatching>
</advancedsettings>
Edit the "Anime" in the code above to the name of your source folder. This regex should handle multi-episode files.

Thanks to gates' post.


Ok, I am trying to get my AniDB scraper working, and I have down the above for my advanced settings.

I have also named half my anime using the anidb own file renamer script.

An example of how my files are named now is : Code Geass Hangyaku no Lelouch - 04 - His Name is Zero - [gg](49715425)

An Example of how My files used to be named ( and work with TVDB) is : K-ON!.S01E08.Freshman Reception!

Now with the new Regex, Code Geass for example with the new episode name setup does not show any episodes at all, all my anime wit the old setup ( S00e00) show up the episodes but with about 10 of the same episode ove rand over.

Why is this not working?

Also another problem I am having is it is refusing to use Banners, no matter how many times I tell it to in the settings. And TVDB does have the banners as before I changed to AniDB they all had banners perfectly.

I have noticed the banners options keeps deactivating in anidb options and I have to keep re-enabling it. It just does not want to stay on or get any banners...

Only anime it has got a banner for is Code Geass, which is also the one anime I am trying the new name setup on and is showing 0 episodes.
(This post was last modified: 2012-04-01 23:38 by Reser.)
find quote
Reser Offline
Junior Member
Posts: 5
Joined: Sep 2011
Reputation: 0
Post: #409
I finally got it working by using Regex of

<regexp> - ([0-9]+)</regexp>

However anidb plugin even with banners enabled in settings still REFUSES to download banners, and I am left with Large Posters SQUASHED into banner size.

Really driving me nuts now..
find quote
bambi73 Offline
Senior Member
Posts: 217
Joined: Jan 2010
Reputation: 0
Location: Czech Republic
Post: #410
@Reser: When you enable banners it doesn't mean they will be default automatically. If banner exist for anime it's only added as another possibility, so you can choose it from Info window. Problem with banners is that they don't exists for each anime, so I did it working this way. Additionally default poster is always from anidb.net and make default something from thetvdb.com, like posters, will make scraper handling even more complicated than it's now.
find quote
Reser Offline
Junior Member
Posts: 5
Joined: Sep 2011
Reputation: 0
Post: #411
(2012-04-01 23:33)Reser Wrote:  
(2012-03-13 17:32)shinner Wrote:  Your anime names are originally absolute numbering right? eg. Naruto 305, not S15E02 or something? If so, try to use regex, it's much less trouble.
Now, create an advancedsettings.xml file, and add this in:

Code:
<advancedsettings>
    <tvshowmatching action = "prepend">
        <regexp>Anime[\\/].*[/\._ \-]()([0-9]+)(-[0-9]+)?</regexp>
     </tvshowmatching>
</advancedsettings>
Edit the "Anime" in the code above to the name of your source folder. This regex should handle multi-episode files.

Thanks to gates' post.


Ok, I am trying to get my AniDB scraper working, and I have down the above for my advanced settings.

I have also named half my anime using the anidb own file renamer script.

An example of how my files are named now is : Code Geass Hangyaku no Lelouch - 04 - His Name is Zero - [gg](49715425)

An Example of how My files used to be named ( and work with TVDB) is : K-ON!.S01E08.Freshman Reception!

Now with the new Regex, Code Geass for example with the new episode name setup does not show any episodes at all, all my anime wit the old setup ( S00e00) show up the episodes but with about 10 of the same episode ove rand over.

Why is this not working?

Also another problem I am having is it is refusing to use Banners, no matter how many times I tell it to in the settings. And TVDB does have the banners as before I changed to AniDB they all had banners perfectly.

I have noticed the banners options keeps deactivating in anidb options and I have to keep re-enabling it. It just does not want to stay on or get any banners...

Only anime it has got a banner for is Code Geass, which is also the one anime I am trying the new name setup on and is showing 0 episodes.

(2012-04-02 23:22)bambi73 Wrote:  @Reser: When you enable banners it doesn't mean they will be default automatically. If banner exist for anime it's only added as another possibility, so you can choose it from Info window. Problem with banners is that they don't exists for each anime, so I did it working this way. Additionally default poster is always from anidb.net and make default something from thetvdb.com, like posters, will make scraper handling even more complicated than it's now.

Erg. If it will not use Banners as priority then the scraper is useless to me.

Guess I will go back to TVDB, it works with the new regex, just having some problems with seasons that have different names E.G Kon! and Kon!! and Tsubasa Chronices. Not sure how to set those out with TVDB,
(This post was last modified: 2012-04-03 00:07 by Reser.)
find quote
shinner Offline
Member
Posts: 52
Joined: Jan 2012
Reputation: 2
Post: #412
Quote:Guess I will go back to TVDB, it works with the new regex, just having some problems with seasons that have different names E.G Kon! and Kon!! and Tsubasa Chronices. Not sure how to set those out with TVDB.
With AniDB, they're considered as separate shows.
With TVDB, you'll have to treat both as just one show, i.e. K-On!! is season 2 of K-On! (so K-On!! episode names should be K-On! S02Exx. Or enable the absolute numbering option.)
(This post was last modified: 2012-04-03 15:45 by shinner.)
find quote
Krumel Offline
Junior Member
Posts: 3
Joined: Apr 2012
Reputation: 0
Post: #413
I currently got a proplem with the scraper.
It doesn't even find the show. It starts to scrape and then immediatly stops.
Also i am not banned from AniDB.
Tricks like downloading the files manually and using them locally didn't work.

Any help or even only to know the cause would be very much appreciated.
find quote
bambi73 Offline
Senior Member
Posts: 217
Joined: Jan 2010
Reputation: 0
Location: Czech Republic
Post: #414
Post your debug log.
find quote
Krumel Offline
Junior Member
Posts: 3
Joined: Apr 2012
Reputation: 0
Post: #415
Very well.
There are quite a lot of errors, but those probably arise from using an experimental build for hi10p support (it doesn't work with the official build either).
I have previously looked at the log, but to me it seems as if the scraper is just working, without an error.
Well, I won't interpret to much into it, as I don't know to much about the plugin.

There you go: http://pastebin.com/G1WGAf9j
find quote
bambi73 Offline
Senior Member
Posts: 217
Joined: Jan 2010
Reputation: 0
Location: Czech Republic
Post: #416
(2012-04-19 23:08)Krumel Wrote:  Very well.
There are quite a lot of errors, but those probably arise from using an experimental build for hi10p support (it doesn't work with the official build either).
I have previously looked at the log, but to me it seems as if the scraper is just working, without an error.
Well, I won't interpret to much into it, as I don't know to much about the plugin.

There you go: http://pastebin.com/G1WGAf9j
Of course not sure, but to me it looks like your installation is complete mess. It's hard to say which messages belongs to application start and which to unsuccesful VideoInfoScanner (next time please let XBMC completely start and then start scanning). But one thing is clear, it never got to scraper. Maybe you should try to delete and recreate your sources again.
find quote
Krumel Offline
Junior Member
Posts: 3
Joined: Apr 2012
Reputation: 0
Post: #417
(2012-04-19 23:28)bambi73 Wrote:  (...) Maybe you should try to delete and recreate your sources again.

That did the job. I don't know why, but I am grateful.
Is there any special reason that recreating the sources works?
It might help me avoid any future errors.
find quote
bambi73 Offline
Senior Member
Posts: 217
Joined: Jan 2010
Reputation: 0
Location: Czech Republic
Post: #418
Quote:2.0.0:
Added: Option to choose scraped rating (Average or Weighted)
Added: Option to choose order of posters and banners (Posters first or Banner first)
Added: Option to choose scraper anime id (from AniDB.net or from TheTVDB.com)
Added: Catching Banned error message from AniDB.net, scraper will output clear warning message instead of anime plot
Changed: Slightly improved studios name filtering

#WARNING: There was some internal INCOMPATIBLE changes to scraper setting. Please check your scraper/source settings if everything is correctly set

anime-list.xml should be updated too. Big thanks to scudlee, his list filled a lot of holes Smile
(2012-04-20 15:21)Krumel Wrote:  That did the job. I don't know why, but I am grateful.
Is there any special reason that recreating the sources works?
It might help me avoid any future errors.
I only guessed from your log (it doesn't found any files). There can be various reasons which depends on your environment.
- maybe you have IP of remote file server in your source url and it was changed by DHCP
- maybe path to your source url changed
- etc. etc.
Let say, something changed, you must find what Smile
Sorry can't help more Smile
(This post was last modified: 2012-04-22 20:33 by bambi73.)
find quote
lad1337 Offline
Junior Member
Posts: 19
Joined: May 2010
Reputation: 0
Post: #419
(2012-04-22 20:28)bambi73 Wrote:  anime-list.xml should be updated too. Big thanks to scudlee, his list filled a lot of holes Smile

i know the addon is in the xbmc repo and everything ... but is i possible to download the source ?
or where is it hosted ? ... i am specifically interested in the anime-list.xml

i found it months ago but now i just dont seam to find it on the interwebs ^^

thanks

lad1337
find quote
lianthus Offline
Junior Member
Posts: 5
Joined: Apr 2012
Reputation: 0
Post: #420
Hey guys, I've been scouring the internets for far too many hours and decided to try to post here. I'm looking to get the AniDB scraper to retrieve 'episode information' for the opening and closing themes for anime. I have a lot of creditless openings and would like to connect it with anidb.net. Also, I have some of the trailers, which they also have info for, that I would let to get as well. If anyone can help me it would be greatly appreciated.
(This post was last modified: 2012-04-23 18:32 by lianthus.)
find quote
Post Reply