[Release] TheAudioDb.com Music Video Scraper

  Thread Rating:
  • 3 Votes - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
olympia Offline
Team-Kodi Member
Posts: 2,499
Joined: May 2008
Reputation: 32
Post: #31
Thanks Pulfer, I will try to look at these during the long weekend.

The upstream for the repos:
http://xbmc.git.sourceforge.net/git/gitweb-index.cgi
find quote
Martijn Offline
Team Kodi
Posts: 11,240
Joined: Jul 2011
Reputation: 162
Location: Dawn of time
Post: #32
(2013-03-29 10:14)olympia Wrote:  Thanks Pulfer, I will try to look at these during the long weekend.

The upstream for the repos:
http://xbmc.git.sourceforge.net/git/gitweb-index.cgi
do note you cannot do pull request to that repo so only way atm is using pastebin for the code.

Always read the XBMC online-manual, FAQ and search the forums before posting.
Do NOT e-mail Team-XBMC members asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting, make sure you read this first

For your mediacenter artwork go to
[Image: fanarttv.png]
find quote
Pulfer Offline
Junior Member
Posts: 19
Joined: Mar 2013
Reputation: 0
Post: #33
(2013-03-29 10:15)Martijn Wrote:  do note you cannot do pull request to that repo so only way atm is using pastebin for the code.

Speaking of music videos, I have one more suggestion. This time related not to scrapers directly but to browsing music videos via UPnP. Currently they are displayed like just "%name". While "%artist - %name" format is much more useful.

For example, when I browse from my PS3
XBMC (UPnP) -> Music Videos -> Genres -> Black Metal

I expect to see
[artwork] 1349 - Sculptor of Flesh
[artwork] Ancient - God Loves the Dead
etc

But current result is:
[artwork] Sculptor of Flesh
[artwork] God Loves the Dead
etc

Here is a simple patch to change result format to "%artist - %name" for music videos:
http://pastebin.com/M1jdZPyi
find quote
Pulfer Offline
Junior Member
Posts: 19
Joined: Mar 2013
Reputation: 0
Post: #34
(2013-03-29 08:18)Pulfer Wrote:  7 -> 8:
repeat="yes" -> repeat="no"

Because it's better to use first found result (= first album released with the track). For example, when we request Accept - Balls To The Wall, we expect to get Balls To The Wall 1983 album. But when repeat is set to "yes", we get Staying a life 1990 album instead.

BTW, is there any way to reverse results order instead of stopping after the first found match? While we really should prefer first match, other matches are used for manual selection in Music Video's info (I didn't know about that earlier).
find quote
scudlee Offline
Team-Kodi Member
Posts: 839
Joined: Jul 2011
Reputation: 51
Post: #35
If you want to trust the returned order of the search results rather than XBMC's, just add sorted="yes" to the results tag in GetSearchResults.

i.e.
Code:
output="<results sorted="yes">\1</results>"

(And leave the repeat="yes" alone.)
find quote
Pulfer Offline
Junior Member
Posts: 19
Joined: Mar 2013
Reputation: 0
Post: #36
(2013-03-30 16:51)scudlee Wrote:  If you want to trust the returned order of the search results rather than XBMC's, just add sorted="yes" to the results tag in GetSearchResults.

i.e.
Code:
output="<results sorted="yes">\1</results>"

(And leave the repeat="yes" alone.)

Thanx a lot, that's what I needed. Blush Works fine with TheAudioDb.com search results.
find quote
olympia Offline
Team-Kodi Member
Posts: 2,499
Joined: May 2008
Reputation: 32
Post: #37
Version 1.1.0 is out

Changelog:
  • Workaround: Scraper won't return 'Unable to Connect to Remote Server' in case of unexpected folder/file naming conventions
  • Added: Music Video Director
  • Added: Music Video Studio
  • Changed: Scrape Music Video screenshot instead of album thumb if available
  • Changed: Trust search result ordering from TheAudioDb.com (Cheers Pulfer and scudlee)
  • Changed: Scrape Album genre instead of track genre for now (Cheers Pulfer - implemented on another way)
  • Added: Scrape Album Year (Cheers Pulfer - implemented on another way)
find quote
Powerhouse Offline
Senior Member
Posts: 142
Joined: Mar 2013
Reputation: 2
Post: #38
Is there an option in theAudioDb.com Music Video Scraper, somewhere, which will allow an NFO and Video Screenshot (or album thumb) to be saved in the same location as the Music Video?

As it is now, I've been manually adding an NFO and Album Artwork to each Music Video (which is quite time consuming with large library's).

Also, on a possible related subject, wouldn't it be easier to have the Music Videos stored within the Music Album's folder, and all the album artwork, fanart, etc could be read from that folder?
find quote
Pulfer Offline
Junior Member
Posts: 19
Joined: Mar 2013
Reputation: 0
Post: #39
(2013-04-03 16:38)olympia Wrote:  Version 1.1.0 is out

Changelog:
  • Workaround: Scraper won't return 'Unable to Connect to Remote Server' in case of unexpected folder/file naming conventions
  • Added: Music Video Director
  • Added: Music Video Studio
  • Changed: Scrape Music Video screenshot instead of album thumb if available
  • Changed: Trust search result ordering from TheAudioDb.com (Cheers Pulfer and scudlee)
  • Changed: Scrape Album genre instead of track genre for now (Cheers Pulfer - implemented on another way)
  • Added: Scrape Album Year (Cheers Pulfer - implemented on another way)

Wow, thanx Blush

New version seems to work like a charm.
find quote
Domin Offline
Junior Member
Posts: 10
Joined: Nov 2010
Reputation: 0
Location: Denmark
Post: #40
(2013-04-03 16:38)olympia Wrote:  Version 1.1.0 is out

Changelog:
  • Workaround: Scraper won't return 'Unable to Connect to Remote Server' in case of unexpected folder/file naming conventions

Very nice work, however i stil get the unable to connect to remote server, on several folders.
Mostly it looks like the error is when filenames/folders are made with a _ instead of a space, which a lot of my folders are.
I got a huge mvid collection (around 900), and if it is of any interest i can provide a list of the file name/ folder names in a .txt file to you.

I included a link to a paste of the debug log, start from the bottom of the log as the debug function have not been enabled all the time:

http://sprunge.us/SdJZ

Here is a short paste of one of the errors:

Code:
11:10:13 T:140047646856960   DEBUG: VideoInfoScanner: Scanning dir 'nfs://192.168.1.109/volume1/video/New/Yoav-Beautiful_Lie_(RAMP_2009)-x264-2009-SRP/' as not in the database
11:10:13 T:140048884741952   DEBUG: LIRC: Update - NEW at 2132010:160 0 KEY_OK_UP devinput (KEY_OK_UP)
11:10:13 T:140047646856960   DEBUG: CNFSFile::Open - opened volume1/video/New/Yoav-Beautiful_Lie_(RAMP_2009)-x264-2009-SRP/yoav-beautiful_lie_(ramp_2009)-x264-2009-srp.nfo
11:10:13 T:140047646856960   DEBUG: CNFSFile::Close closing file volume1/video/New/Yoav-Beautiful_Lie_(RAMP_2009)-x264-2009-SRP/yoav-beautiful_lie_(ramp_2009)-x264-2009-srp.nfo
11:10:13 T:140047646856960   ERROR: Parse: Could not find scraper function NfoUrl
11:10:13 T:140047646856960   DEBUG: FindMovie: Searching for 'Yoav Beautiful Lie (RAMP 2009)' using TheAudioDb.com for Music Videos scraper (path: '/storage/.xbmc/addons/metadata.musicvideos.theaudiodb.com', content: 'musicvideos', version: '1.1.0')
11:10:13 T:140047646856960   DEBUG: scraper: CreateSearchUrl returned 2009
11:10:13 T:140047646856960   ERROR: Run: Unable to parse XML
11:10:13 T:140048884741952   DEBUG: ------ Window Init (DialogYesNo.xml) ------
11:10:14 T:140048884741952   DEBUG: LIRC: Update - NEW at 2132901:69 0 KEY_LEFT devinput (KEY_LEFT)
11:10:14 T:140048884741952   DEBUG: OnKey: 169 (a9) pressed, action is Left
11:10:14 T:140048884741952   DEBUG: LIRC: Update - NEW at 2133151:69 0 KEY_LEFT_UP devinput (KEY_LEFT_UP)
11:10:14 T:140048884741952   DEBUG: LIRC: Update - NEW at 2133251:160 0 KEY_OK devinput (KEY_OK)
11:10:14 T:140048884741952   DEBUG: OnKey: 11 (0b) pressed, action is Select
11:10:14 T:140048884741952   DEBUG: ------ Window Deinit (DialogYesNo.xml) ------
11:10:14 T:140047646856960 WARNING: No information found for item 'nfs://192.168.1.109/volume1/video/New/Yoav-Beautiful_Lie_(RAMP_2009)-x264-2009-SRP/yoav-beautiful_lie_(ramp_2009)-x264-2009-srp.mkv', it won't be added to the library.
11:10:14 T:140047646856960   DEBUG: VideoInfoScanner: No (new) information was found in dir nfs://192.168.1.109/volume1/video/New/Yoav-Beautiful_Lie_(RAMP_2009)-x264-2009-SRP/
11:10:14 T:140048884741952   DEBUG: LIRC: Update - NEW at 2133496:160 0 KEY_OK_UP devinput (KEY_OK_UP)

Regards
Domin

ASRock ION 330HT / Raspberry PI
(This post was last modified: 2013-04-04 11:21 by Domin.)
find quote
olympia Offline
Team-Kodi Member
Posts: 2,499
Joined: May 2008
Reputation: 32
Post: #41
(2013-04-04 11:05)Domin Wrote:  Very nice work, however i stil get the unable to connect to remote server, on several folders.
Mostly it looks like the error is when filenames/folders are made with a _ instead of a space, which a lot of my folders are.
I got a huge mvid collection (around 900), and if it is of any interest i can provide a list of the file name/ folder names in a .txt file to you.

Let's start with only one example Folder/File name what is ending up with the error message.
find quote
Domin Offline
Junior Member
Posts: 10
Joined: Nov 2010
Reputation: 0
Location: Denmark
Post: #42
I have edit my post to include debug info, both a quick view in the post and the entire debug log on sprunge.us.

Thanks for looking a this, its much appreciated.

Regards
Domin

ASRock ION 330HT / Raspberry PI
find quote
zag Offline
Team-Kodi Member
Posts: 1,685
Joined: Oct 2007
Reputation: 20
Location: UK
Post: #43
Artist "YOAV" does not even exist on theaudiodb.com, so that would be the first obvious thing to fix...

Also I noticed the filename has some additional brackets "(ram_2009)".

Taking a quick look at music brainz, the song title should simply be "Beautiful Lie".

http://musicbrainz.org/recording/aa7cb01...b938f48250

Try fixing both those things and seeing how it goes.

HTPC - XBMC Gotham, OpenELEC, Harmony Smart Remote, Intel Haswell NUC, 40gb intel SSD, Core i3, 4gb RAM
Storage - 2 x qnap 8tb 419p+ NAS
Display LG 46" LCD + Casio Bulbless projector [PICS]
[Image: widget]
(This post was last modified: 2013-04-04 12:13 by zag.)
find quote
Domin Offline
Junior Member
Posts: 10
Joined: Nov 2010
Reputation: 0
Location: Denmark
Post: #44
Still the problem here is that i get a pop up saying the server is unavailable, that means i have to click like 300 times for files that do not exist on theaudiodb.com, it should just skip those and not give a pop up saying the server is not responding.

Perhaps a better description of the actual error and a possibility to select do not ask again in this search would be a way of fixing this and still let people see there are files that are not found.

Also the RAM 2009 tag is there because its a live recording from the RAM 2009 festival, and its a rather standard way of tagging live audio/video.

I will start adding stuff to theaudiodb.com, however i would like this "error" fixed still.

Regards
Domin

ASRock ION 330HT / Raspberry PI
(This post was last modified: 2013-04-04 12:22 by Domin.)
find quote
olympia Offline
Team-Kodi Member
Posts: 2,499
Joined: May 2008
Reputation: 32
Post: #45
Well, you can always have a workaround:
http://wiki.xbmc.org/?title=advancedsett...scanner.3E

...actually this is an easy fix. Expect v1.1.1 to address the issue with 'Yoav-Beautiful_Lie_(RAMP_2009)-x264-2009-SRP' popping up the server unavailable error out in 2-3 hours when I can push the fix.
(This post was last modified: 2013-04-04 13:33 by olympia.)
find quote
Post Reply