[WIP] AniDB.net Anime Video Scraper

  Thread Rating:
  • 3 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
« Next Oldest | Next Newest»
bambi73 Offline
Senior Member
 Posts: 165
Joined: Jan 2010
Reputation: 0
Location: Czech Republic
2012-06-07 09:30 Post: #441
@Kupo, Sodden, vask: AniDB.net scraper uses Http API, but I guess it apply for both. You can check if API is up by opening

http://api.anidb.net:9001/httpapi?reques...r=1&aid=98

in your browser (aid is anime id). And looks like it's up now.
find quote
Pawn Offline
Junior Member
 Posts: 26
Joined: Mar 2011
Reputation: 0
2012-06-09 11:07 Post: #442
(2012-06-07 09:30)bambi73 Wrote:  @Kupo, Sodden, vask: AniDB.net scraper uses Http API, but I guess it apply for both. You can check if API is up by opening

http://api.anidb.net:9001/httpapi?reques...r=1&aid=98

in your browser (aid is anime id). And looks like it's up now.

So... if it says "<error>Banned</error>" there I reckon it's not working? I'm not sure why I would have been banned...
find quote
shinner Offline
Member
 Posts: 52
Joined: Jan 2012
Reputation: 2
2012-06-11 15:07 Post: #443
(2012-06-09 11:07)Pawn Wrote:  
(2012-06-07 09:30)bambi73 Wrote:  @Kupo, Sodden, vask: AniDB.net scraper uses Http API, but I guess it apply for both. You can check if API is up by opening

http://api.anidb.net:9001/httpapi?reques...r=1&aid=98

in your browser (aid is anime id). And looks like it's up now.

So... if it says "<error>Banned</error>" there I reckon it's not working? I'm not sure why I would have been banned...

Maybe you scraped too much at once. Wait 24h and try again.
find quote
xSoul Offline
Junior Member
 Posts: 2
Joined: Jul 2012
Reputation: 0
2012-07-20 14:41 Post: #444
Since two days I'm getting this error: "Unable to connect to remote server. Would you like to continue scanning?"


Somehow it won't scrape anymore Confused
Does anyone knows the problem, because I can still view the XML page.
So I'm not banned or something for overflooding the servers.

Here's the debug
Code:
15:01:34 T:3064  NOTICE: VideoInfoScanner: Starting scan ..
15:01:34 T:3064   DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'D:\Animes\Fullmetal Alchemist Brotherhood\'
15:01:34 T:3064   DEBUG: ADDON::CScraper::FindMovie: Searching for 'Fullmetal Alchemist Brotherhood' using AniDB.net scraper (path: 'C:\Users\#####\AppData\Roaming\XBMC\addons\metadata.anidb.net', content: 'tvshows', version: '2.0.0')
15:01:34 T:3064   DEBUG: scraper: CreateSearchUrl returned <url gzip="yes" cache="anidb.xml">http://sites.google.com/site/anidblist/anidb.xml</url>
15:01:34 T:3064   DEBUG: Using proxy http://:8080
15:01:34 T:3064   DEBUG: FileCurl::Open(0678ECB4) http://sites.google.com/site/anidblist/anidb.xml
15:01:34 T:3064    INFO: XCURL::DllLibCurlGlobal::easy_aquire - Created session to http://sites.google.com
15:01:35 T:3064 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 56
15:01:35 T:3064   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
15:01:35 T:3064   ERROR: ADDON::CScraper::Run: Unable to parse web site
(This post was last modified: 2012-07-20 15:04 by xSoul.)
find quote
zosky Offline
Donor
 Posts: 302
Joined: Dec 2008
Reputation: 1
Location: toronto. canada
2012-07-20 19:52 Post: #445
hi yall.

Gintama (the original) > aniDB:3468
Gintama' (the squeal) > aniDB:8126

adding a <qoute> is a stupid way to name a sequal... but that's what we have here
what does one need to do so both are *not* picked up as the original
8 xbmc frodo's (rc3) ... atom/ion(xbmcBuntu) | amd3000/nVidia9800(mint12) | iphone4(5.1.1) | hpTouchPad(cyanogen9) ... and more
+ central mySQL db + 11TB mdadm raid6 (+transmission-daemon & flexGet)
+ maraschino + IRtoy.v2
[Image: widget]
find quote
scudlee Offline
Team-XBMC Member
 Posts: 582
Joined: Jul 2011
Reputation: 45
2012-07-21 00:15 Post: #446
(2012-07-20 19:52)zosky Wrote:  hi yall.

Gintama (the original) > aniDB:3468
Gintama' (the squeal) > aniDB:8126

adding a <qoute> is a stupid way to name a sequal... but that's what we have here
what does one need to do so both are *not* picked up as the original
Two options:
a) Simply manually refresh the Gintama' folder and select the right series. (Navigate to the folder in Files view, press 'i' and select Refresh.)

2. Create a tvshow.nfo file containing "aid=8126" in the Gintama' folder (technically, you should put in the full anidb url as linked in your post, but that part is all you really need). You'll still need to manually update as above to fix it, but if you ever need to automatically scan in again, it should work straight away without needing to be manually fixed.
find quote
KoldPT Offline
Junior Member
 Posts: 1
Joined: Jul 2012
Reputation: 0
2012-07-29 01:40 Post: #447
Hey, I've been having some issues getting anime to work on XBMC.

I've tried it on several different computers, with several different naming schemes and even using a complex as hell advancedsettings file, and yet it hasn't worked so far Tongue

Right now my setup is like this:

S:\shares\anime\Cowboy Bebop
and under that
Cowboy Bebop - 21 - Boogie-Woogie Feng-Shui
(...)

S:\shares\anime\Eureka Seven Ao
Eureka Seven Ao - 01 - Born Slippy (Deep Blue)
(...)

This is the same naming convention i use for my TV Shows, using FileBot. (Retro Game Master - 2x02 - Challenger; Black Mirror - 1x01 - The National Anthem...).

I've tried several others, and yet XBMC refuses to let me watch anime with the fancy features! It picks up banners and whatnot, but then just explodes. Last time, it even started recognizing episodes wrong... Help?

https://dl.dropbox.com/u/14752045/xbmc.log - my most recent log file. Opened XBMC up, it refreshed library, nothing. De-set content, set it back up and had it refresh files, still nothing.

edit: Just switched to using some regexps posted in the earlier thread and the anidb renamer... That'll have to do.
(This post was last modified: 2012-07-29 10:47 by KoldPT.)
find quote
aelfwyne Offline
Junior Member
 Posts: 10
Joined: Aug 2012
Reputation: 0
2012-08-26 02:44 Post: #448
Still having "issues" with this. A lot of shows are not recognized even though they are reasonably named, with a directory that ONLY contains the title of the show and filenames that have been carefully formatted.

For example, the anime "Aika"...

Anidb URL: http://anidb.net/perl-bin/animedb.pl?show=anime&aid=280
Not recognized at all with a folder named "Aika" or "Agent Aika", so I created a tvshow.nfo file with the anidb URL in it as instructed above.

Filenames are as follows:
Aika - 1 - Beautiful Agent - [1997-1999][anime-takeover][394A72BA].ogm
Aika - 2 - Naked Mission - [1997-1999][anime-takeover][F63F2BB9].ogm
..etc etc

This is a standard format applied via Anidb'O'Matic that works for about 85% of my shows, but about 15% it fails.

With tvshow.nfo, it gets the series information -- but not episode details.
Obviously the name is there, there's only ONE show by that name, so why in blue blazes is the scraper not picking up episode details? What's it take to get this thing to work?

Logfile below (cleaned up a bit and hopefully all the relevant bits)

Code:
19:37:07 T:2725125008   DEBUG: CFileSMB::Open - opened Anime C/Aika/tvshow.nfo, fd=10000
19:37:07 T:2725125008   DEBUG: CFileSMB::Close closing fd 10000
19:37:07 T:2725125008   DEBUG: scraper: NfoUrl returned <url gzip="yes" cache="280.xml">http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280</url>
19:37:07 T:2725125008   DEBUG: VideoInfoScanner: Found matching URL NFO file: smb://EDI/Anime C/Aika/tvshow.nfo
19:37:07 T:2725125008   DEBUG: VideoInfoScanner: Fetching url 'http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280' using AniDB.net scraper (content: 'tvshows')
19:37:07 T:2725125008   DEBUG: GetVideoDetails: Reading movie 'http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280' using AniDB.net scraper (file: '/root/.xbmc/addons/metadata.anidb.net', content: 'tvshows', version: '2.0.0')
19:37:07 T:2725125008   DEBUG: FileCurl::Open(0xa26e0a04) http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280
19:37:07 T:2725125008    INFO: easy_aquire - Created session to http://api.anidb.net
19:37:07 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:07 T:2725125008   DEBUG: scraper: GetDetails returned <details><url function="GetDetailsExt" cache="anime-list.xml">http://sites.google.com/site/anidblist/anime-list.xml</url></details>
19:37:07 T:2725125008   DEBUG: FileCurl::Open(0xa26e0a04) http://sites.google.com/site/anidblist/anime-list.xml
19:37:09 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:10 T:2725125008   DEBUG: scraper: GetDetailsExt returned <details><id>280</id><title>Aika</title><year>1997</year><premiered>1997-04-25</premiered><votes>1530</votes><rating>5.98</rating><genre>Ecchi</genre><genre>Pantsu</genre><plot>A catastrophic earthquake has left Tokyo, and most of Earth for that matter, under the sea level. Aika is a salvager, who retrieves various remains from the watery ruins. When Aika accepts the dangerous mission to locate and retrieve the mysterious Lagu, she discovers that other dangerous parties are also interested.</plot><actor><name>Sakuma Rei</name><role>Sumeragi Aika</role><thumb>http://img7.anidb.net/pics/anime/23759.jpg</thumb></actor><actor><name>Konishi Hiroko</name><role>Aida Rion</role><thumb>http://img7.anidb.net/pics/anime/25984.jpg</thumb></actor><actor><name>Takayama Minami</name><role>Bianca</role><thumb>http://img7.anidb.net/pics/anime/46104.jpg</thumb></actor><credits>Studio Fantasia</credits><credits>Yamauchi Noriyasu</credits><director>Nishijima Katsuhiko</director><studio>Studio Fantasia</studio><episodeguide><url gzip="yes" cache="280.xml">http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280</url></episodeguide><url function="GetFanartDataAPI" cache="tvdb-280.xml">http://www.thetvdb.com/api/1D62F2F90030C444/series/80900/all/en.zip</url></details>
19:37:10 T:2725125008   DEBUG: FileCurl::Open(0xa26e0a04) http://www.thetvdb.com/api/1D62F2F90030C444/series/80900/all/en.zip
19:37:10 T:2725125008   DEBUG: scraper: GetFanartDataAPI returned <details><url function="GetFanartDataReturnAPI" cache="280.xml">&</url></details>
19:37:10 T:2725125008   DEBUG: scraper: GetFanartDataReturnAPI returned <details><thumb>http://img7.anidb.net/pics/anime/5721.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-4.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-1.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-2.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-3.jpg</thumb><fanart url="http://thetvdb.com/banners/"><thumb dim="1280x720" colors="|25,35,37|25,35,37|25,35,37|" preview="_cache/fanart/original/80900-2.jpg">fanart/original/80900-2.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-1.jpg">fanart/original/80900-1.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-3.jpg">fanart/original/80900-3.jpg</thumb><thumb dim="1920x1080" colors="" preview="_cache/fanart/original/80900-6.jpg">fanart/original/80900-6.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-8.jpg">fanart/original/80900-8.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-7.jpg">fanart/original/80900-7.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-9.jpg">fanart/original/80900-9.jpg</thumb><thumb dim="1920x1080" colors="" preview="_cache/fanart/original/80900-4.jpg">fanart/original/80900-4.jpg</thumb><thumb dim="1920x1080" colors="" preview="_cache/fanart/original/80900-5.jpg">fanart/original/80900-5.jpg</thumb></fanart></details>
19:37:10 T:2725125008   DEBUG: VideoInfoScanner: Adding new item to tvshows:smb://EDI/Anime C/Aika/
19:37:11 T:2725125008   DEBUG: OpenDir - Using authentication url xxxxxxxxxxxxxxxxxxxxxxx erased
19:37:11 T:2725125008    INFO: Caching image from: http://thetvdb.com/banners/fanart/original/80900-2.jpg to special://masterprofile/Thumbnails/Video/Fanart/4e1c3bb1.tbn
19:37:11 T:2725125008   DEBUG: FileCurl::Open(0xb10c4168) http://thetvdb.com/banners/fanart/original/80900-2.jpg
19:37:11 T:2725125008    INFO: easy_aquire - Created session to http://thetvdb.com
19:37:11 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:13 T:2725125008    INFO: Caching image from: http://img7.anidb.net/pics/anime/5721.jpg to special://masterprofile/Thumbnails/Video/4/4e1c3bb1.tbn with width 512 and height 512
19:37:13 T:2725125008   DEBUG: SECTION:LoadDLL(special://xbmcbin/system/ImageLib-i486-linux.so)
19:37:13 T:2725125008   DEBUG: Loading: /usr/local/lib/xbmc/system/ImageLib-i486-linux.so
19:37:13 T:2725125008   DEBUG: FileCurl::Open(0xa26e09ac) http://img7.anidb.net/pics/anime/5721.jpg
19:37:13 T:2725125008    INFO: easy_aquire - Created session to http://img7.anidb.net
19:37:13 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:14 T:2725125008   DEBUG: CAnnouncementManager - Announcement: OnUpdate from xbmc
19:37:14 T:2725125008   DEBUG: GOT ANNOUNCEMENT, type: 8, from xbmc, message OnUpdate
19:37:14 T:2725125008   DEBUG: OpenDir - Using authentication url xxxxxxxxxxxxxxxxxxxxxxx erased
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 1 - Beautiful Agent - [1997-1999][anime-takeover][394A72BA].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 2 - Naked Mission - [1997-1999][anime-takeover][F63F2BB9].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 3 - Takeoff Position - [1997-1999][anime-takeover][6082355A].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 4 - A Flower Blooming in Space - [1997-1999][anime-takeover][88CD34D0].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 5 - The Golden Delmo Operation - [1997-1999][anime-takeover][D71BEC74].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 6 - Delmo Operation  White Silver - [1997-1999][anime-takeover][79D6FCA6].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 7 - Break the Decisive Battle! Delmo Base! - [1997-1999][anime-takeover][BB3EB663].ogm
19:37:14 T:2725125008  NOTICE: CleanDatabase: Starting videodatabase cleanup ..

Looks like it (as I thought) actually finds the URL I passed it... but it REFUSES to recognize any of the episodes! Is it bouncing back to thetvdb instead of anidb for episode details? It should NOT be doing that, but it looks like what it's doing.
(This post was last modified: 2012-08-26 02:53 by aelfwyne.)
find quote
pigers Offline
Junior Member
 Posts: 1
Joined: Aug 2012
Reputation: 0
2012-08-27 22:43 Post: #449
Hi
im using xbmc-12.0-Frodo_alpha4, everything works fine until i try to scrape some anime from my disk. For example: let's take "Accel World", anidb ID is 8691 ... and I get message saying that im banned.
After doing some digging i got to information if i can get XML from AniDB api via my browser then i should get info.
So i can see that XML - but XBMC can't.
Any workarounds ?

File tree can be found here.
Log is here.

Regards
pigers
(This post was last modified: 2012-08-27 22:54 by pigers.)
find quote
akovia Offline
Junior Member
 Posts: 42
Joined: Aug 2012
Reputation: 1
2012-08-28 02:30 Post: #450
Hi,
New user here and have been reading all day now but still feel a little lost.

I have an extensive anime collection and have created my own filing/naming scheme that is incompatible with xbmc. I'm searching for the best strategy to attack renaming everything with a high probability of success. I don't mind hand editing some stuff to get it accurate, but I would like to start on the right track, so I'm looking for some success stories or a pointer to an updated guide that walks you through and what to expect. Maybe just a good naming scheme that catches most series, movies, and OVAs.

Is Filebot a good option for renaming and is there a successful script for it?

Lastly I did read that there will be no support for OPs & EDs (specials?) which is fine, but I was curious how you deal with them as far as naming and filing as to not disrupt matching. (sub-folder?)

Any help, strategies, or anecdotes would be much appreciated.
find quote
Post Reply
Contact Us | XBMC Home | Return to Top | Return to Content | Lite (Archive) Mode | RSS Syndication