• 1
  • 16
  • 17
  • 18(current)
  • 19
  • 20
  • 71
Release [MOD] AniDB.net scrapers for TV shows and Movies
Pretty much the same:

TV Show
----> Season #1
--------> TV Show - EP1 - Episode Title [CRC32]
----> Season #2
--------> TV Show - EP1 - Episode Title [CRC32]

At first I thought the issue was that the season # was missing from the file names. So I renamed a couple of shows using "S01EP1". When that didn't work, I tired "1x01". Neither of these formats were scrapped. Then I decided to try on my other computer as a comparison. The shows with just EP in the file name were scrapped, but a lot of episodes were repeated since it couldn't differentiate between seasons. The shows that I had renamed with the season # in the file name were scrapped just fine. I think it has to do something with the seasons. On the computer where I'm using the AniDB Mod, I tried scrapping again with some different shows and saw that 3 shows were scrapped. While none of the episodes were recognized, the title of the shows were added to the library. The only thing different about these 3 shows from the rest is that they have only one season. All the episodes are placed in a "Season 1" folder. That's pretty what I was able to figure out.
Reply
I'd suggest looking at Video library/Naming files/TV shows#Files (wiki). Most of your deductions about what works and what doesn't are spot on. The season folders are completely ignored by default.

Why your AniDB computer doesn't work at all... My guess would be that you haven't put an action="prepend" on the tvshowmatching tag. That would cause only the custom regexps to be checked for.

If you could paste a debug log (wiki) onto http://xbmclogs.com it would probably clear things up.
Reply
Was having some issues with xbmclogs so went to pastebin instead. Also had to cut the log about halfway through since it was ridiculously huge, but hopefully it should be more than enough to figure out what's going on.

http://pastebin.ca/2405357

Also, I checked my advancedsettings.xml and action="prepend" is there. The settings are pretty much copied from the first page of this thread except I changed the "s" to "sp" to match my naming convention.
Reply
From what I can see from the log, nothing unexpected is happening. All your files are being dealt with as best they can be, without interference from the anime regexps.

Obviously there are issues, but they're the same ones you should be experiencing on the non-anidb computer, e.g. your Archer folder should be called "Archer (2009)" otherwise it gets picked up as an earlier series, plus the whole "ep1"="season 1, episode 1" thing.

To address the latter, try adding:
Code:
<regexp>[Ss]eason[\._ -]([0-9]+)[\\/][^\\/]*?[\._ -][Ee][Pp]_?([0-9]+(?:(?:[a-i]|\.[1-9])(?![0-9]))?)([^\\/]*)$</regexp>
to your tvshowmatching. (It's basically the default one with a season folder added.)


If there is definitely an instance where the anidb computer definitely behaves differently to the non-anidb one, try to get a log just demonstrating that.
Reply
OK, so there's not really a special instance I can think of that really separates the two, but I seemed to have resolved the issue. In order to further investigate, I backed up my advancedsettings.xml and uninstalled XBMC on my AniDB computer. Then I pretty much scrapped the my non-anime content under the following conditions: deafult, with the advancedsettings.xml, and finally with the test version of the Anidb Mod. In each case I was able to scrape successfully. After that I scrapped my anime and had no issues. No sure what the problem was, but it seemed to go after a clean install.
Reply
Using the anidb script and the regxp settings found in the OP I've renamed and gotten nearly all of my anime collection to scan. However XBMC doesnt seem to pick up any series where a ? was replaced with _ (and for no apparent reason it also wont pick up the Yondemasu Azazel San OVA). I've tried including a .nfo file in the directories but still no luck. Anyone had any success getting shows like this to scrape?
Reply
Not sure what the issue could be. I've (re-)tested a few titles, and they all work.

The default search ignores all punctuation during the search, and then relies on XBMC to find the best match to your original folder name among the results.

You're not using the Google search option are you? I've found that notoriously spotty (and ban-causing). (If you're using 2.0.0rc1, then you're definitely not.)
You don't need it anyway if you're renaming to match anidb.

As ever, a debug log (wiki) would be much appreciated.

Examples tested:
Love Love_: 5 results (a1762, a5296, a2498, a4584, a5364).
Kore wa Zombie Desuka_ Ee, Kore ga Saishuukai Desuka_: 1 result (a8448).
Mondaiji-tachi ga Isekai kara Kuru Sou Desu yo_: 2 results (a9332, a9519).
Shinryaku!_ Ika Musume: 3 results (a7486, a8294, a9046) - in this case the wrong title is picked (as noted in the first post).
Yondemasu yo, Azazel-san: 3 results (a7275, a9501, a8288).
Reply
Thanks for the help, as it was only two or three titles I just ended up making tvshow.nfo's for them which seems to have taken care of it

Edit: Just wondering what the "Enable prequel lookup" setting is for, as far as I can see theres nowhere that shows the links between shows this presumably scrapes?
Reply
(2013-06-08, 22:01)Sesshoumaru Wrote: Hi scudlee

In the meantime I managed to create a very mighty regular expression that can handle about 99% of all my anime files (season and episode).
I debugged XBMCs handling and the information is extracted properly. Unfortunatly, now the next issues comes up - season handling.

Season 1 is always handled properly, but Season 2 or 3 not. On AniDB season 1, 2 and 3 have own ID's (because they are own animes). But the episode guide seems only to be downloaded for season 1 only. Of cause this episode guide contains the information which IDs the sequals of this anime has, it just seems not to be taken into account. A good example is "Boku wa Tomodachi ga Sukunai".

Is this known or should it work?

(2013-06-09, 00:16)scudlee Wrote: @Sesshoumaru: This is known and is one of the main differences between the anidb and tvdb scrapers. Each (named) season should be treated as a separate series in its own right.
So you'd have:
Boku wa Tomodachi ga Sukunai
|---Season 1, Episodes 1-12 + Specials (Season 0)
Boku wa Tomodachi ga Sukunai Next
|---Season 1, Episodes 1-12 + Specials (Season 0)
Boku wa Tomodachi ga Sukunai: Yaminabe wa Bishoujo ga Zannen na Nioi
|---Season 1, Episode 1
Boku wa Tomodachi ga Sukunai: Relay Shousetsu wa Ketsumatsu ga Hanpanai
|---Season 1, Episode 1
(The latter two could also be treated as movies with the movie scraper)

(2013-06-12, 05:02)ZERO &lt;ibis&gt; Wrote: You do realize there are not "seasons" in anime.
Boku wa Tomodachi ga Sukunai and Boku wa Tomodachi ga Sukunai Next are actually different shows, it is right there in the name.

However, as long as your regex is configured correctly and in some cases you will need to use .nfo files you can treat your anime like American cartoons.


Hi,

I was Just thinking it might be great to have an option to enable/disable season when anime are sequel/prequel.

I'm agree with ZERO <ibis> and I totaly understand Sesshoumaru Smile

For me some Anime like "Code Geass: Hangyaku no Lelouch R2" are the season 2 of "Code Geass: Hangyaku no Lelouch"
and not a differents shows.
Plus, it's better for those using watchlist/skin widget (in progress show).

Anyway, I think that would be a great option.
Dont know if it's pain in the ass to code ... but it seems with anime-list-mapping, a lot of the work is already done.

Just a quick word : Thank you scudlee for this Mod.
This is a huge work (2.0 version rocks !). I think we are many anime fans to use it, but you are not reward like you should !
So Thank you again. And a special thx to bambi73 also for its works on the first scapper.

Cheers
Gates
Reply
This is my opinion on seasons:

1. It would be reeeaaalllly hard to do using just the data from AniDB. This is the main blocker, if it was easy, I'd likely do it as an option even though I disagree with it. You'd have to navigate up and down the relationship tree, find the "first" season and then increment the seasons upward from there - paying careful attention to things like inter-season movies/OVAs (which would have to be ignored in the count). That would be very tricky to do (even just "incrementing" is hard with scraper-logic), and then you might have branching sequels...

And that doesn't even take into account what to do with special episodes, which would overlap like crazy...

And the biggest killer would be the impact on AniDB, since you'd have to snatch each potential season's data every time you add an episode. That would be a quick road to Ban Town without adding delays (which would then slow everything to a crawl).

2. "So what about using the mapping list?" The mapping list contains a defaulttvdbseason for each series. You could take the tvdbid from the series, find all the other series with the same id and then get the data just from them and assign each of them the season number from their own defaulttvdbseason and/or individual mapping. You'd still have the issue of polling AniDB too quickly to deal with, but it would be easier code-wise...

My issue there is, if you want to use the season numbering from TVDB, why bother using AniDB in the first place? Just use the TVDB scraper.

You can always make URL .nfos (or mixed .nfos) that point to the show's TVDB page, even in an AniDB-scraped source, and the TVDB scraper will be used instead for just that show. (Or vice versa, AniDB URLs in a TVDB-scraped source - although you may need to uninstall the un-modded scraper to ensure the Mod gets used.)
Reply
Yea I suppose if you wanted to have them treated as the same season you would need to scrap from a non anime related western source b/c those are the only people that would actually list different shows as being the same.

As far as progress tracking goes note that no anime progress tracker treats multiple anime as being all of the same season.

Although I suppose if you want to make things simple you could claim that every anime is really just another season of Haiyore! Nyaruko-san and list them all in there or Lucky Star.

When is V2 finally going to be the release version btw?
Reply
To-do list for 2.0.0 is basically:
Fix one more bug
Add documentation

If I skip @swe-divx's thing (sorry Sad - It will happen soon after), I can probably get it out on Monday. (DO NOT HOLD ME TO THAT)

The documentation (i.e. readmes, changelog, and the forum announce post) is the thing I've been dragging my feet on. I really want to include explanations for all the settings and features, but I keep putting off actually writing it all down.

Note that 2.0.0 is going to be basically what you've already got in the RC. No new features have yet been added (like the genre shuffle and location/source tagging we talked about). They will come afterwards. BUT... I will push them as soon as each feature is written, so expect a flurry of point releases in the coming months. Once the 2.0.0 base is "out there" it will be so much easier to work on new stuff and release in a sensible manner.
Reply
Awesome! This is the reason I have been waiting for v2 to get out so we can start to see some movement on the genre and tagging system!
Reply
scudlee great work! I know this is one of the best things that happened to XBMC for us ho loves Anime! Angel

Im okej if you take your time on the new thing. If you want more help with getting info for the xml so send a note.
Right now im working on a new responsive theme for zPanel, and its SUMMER Big Grin
Reply
New Features in 2.0.0

Season artwork
Although AniDB doesn't use seasons, the scraper now attempts to add artwork for seasons 1 and 0.
This is useful as if you want to link a movie to a TV show, you will be put into a season view when looking at the show.
The season 1 artwork will use the defaulttvdbseason from the mapping list.

Movie sets from anime-movieset-list.xml
Movie sets are now taken by default from anime-movieset-list.xml and not from themoviedb.org.
If you want to continue only using themoviedb.org sets, you will need to change the scraper settings accordingly.
You can also take sets from both sites and choose a preference if there's a conflict.
If you previously disabled sets, that will continue to be respected.
The sets will try to respect "Use Official Title" setting although currently only English language set titles are available (alongside the "main" Romaji titles).
Your help is needed to supply (Official) Non-English set titles.

You can also define a personal movie set list.

Example:
Code:
<anime-set-list>
  <set>
    <anime anidbid="6432">Lupin Sansei vs Meitantei Conan</anime>
    <titles>
      <title>Lupin Sansei Collection</title>
    </titles>
  </set>
</anime-set-list>
In the personal list you don't need to include language information for set titles.
The movie names are ignored by the scraper, only the anidbid is used (they're just there for readability).
To exclude movies from being in any set, leave the title blank.

A simple way to get the personal set list to work (on Windows) is to place the file directly on your C:\ drive, and use the URL:
Code:
file:///c:/pers-anime-movieset-list.xml
(Note the three forward slashes after file:)

Trailers and Certification from themoviedb.org
Thanks to the rewrite these use the shared functions from the official themoviedb/universal scrapers (as do themoviedb.org sets) so they should function in the same way.

There is also the possibility of expanding to use other shared functions (some more easily than others) if anyone has any suggestions.

Misc.
Genre handling has been rewritten, and the old "Return only Genres with weight" has been replaced by "Minimum Genre weight". If you'd previously changed the old setting, you'll need to adjust the new setting accordingly.
Poster/banner settings have been removed, both are now always retrieved when possible.
Google search has also been rewritten to avoid hitting AniDB so much (and potentially causing bans).


The major effort in 2.0.0 was a complete back-end rewrite, which is why its a full version bump despite only a handful of new features.

The AniDB Client rules have also been updated slightly to always pad episode numbers unless the total episode count is under 10, and also to handle trailing periods.
Reply
  • 1
  • 16
  • 17
  • 18(current)
  • 19
  • 20
  • 71

Logout Mark Read Team Forum Stats Members Help
[MOD] AniDB.net scrapers for TV shows and Movies8