[Release] Universal Scraper for Music Albums

  Thread Rating:
  • 5 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
A-Swiss Offline
Donor
Posts: 32
Joined: Sep 2010
Reputation: 0
Location: Switzerland
Smile  RE: [Release] Universal Scraper for Music Albums
Post: #301
For me, there is only one (much time eating) thing, that allways works...
Use an MP3 Tag Editor and add a proper album art into the Music File...
And when it's open, check that all needed fields are filled out properly.
I have spent much time over some years, to get a proper library... and as an advantage - all my music files are now independent from any scraper result, due to proper tags including albumart.

The only thing is have on one installation, that the DB Path is scanned twice... (maybe i pressed ADD to DB twice...)
2/3 of my data is now (what i call) clean.... the rest will be done when in mood.
And i built my folders on the NAS in a way, that only proper files are added into the DB. Each time a album is cleaned, i move it into the proper section.
And with a little time, it works also for compilations... 3260 Album with about 55600 Tracks from more than 4700 artists - i'm happy with XBMC.. :-)

My Equipment:
HTPC (i7 Ivy 3.5 GHz, 16 GB RAM, 256 GB SSD) with Gotham 13.0,
Onkyo TX-NR818, Samsung UE55ES8000, Teufel Theater LT-5, Panasonic DMP-BDT500, Toshiba HD-X1 HD-DVD,
Synology DS1812+ / DX-510 (8x3 TB, 5x3 TB in RAID5+)
(This post was last modified: 2014-05-19 22:19 by A-Swiss.)
find quote
EvilOzzir Offline
Junior Member
Posts: 2
Joined: May 2014
Reputation: 0
Post: #302
I'am having a problem when scrapping albums with special characters like á, é, í, ó, ú. What should I do?
find quote
gagnehj Offline
Junior Member
Posts: 24
Joined: Jun 2014
Reputation: 0
Post: #303
Quick note to say thanks for all the hard work put into the Albums/Artists scraper(s).

I typically use TagScanner to manager my audio file IDTags ... actually posted some templates on the forum for those wanting to create a generic NFO template with TagScanner (http://forum.xbmc.org/showthread.php?tid=197283)

Curious about a DISCOGS integration; I know that quite a while back, they started blocking XBMC-based scrapers as their servers couldn't handle the constant requests. Was this ever resolved and are there any plans in the near future to include their metadata as part of the query?

Cheers,
find quote
el_rodro_83 Offline
Junior Member
Posts: 1
Joined: Apr 2014
Reputation: 0
Post: #304
Hi,
I've been trying to get information for my albums and most work fine using MusicBrainz.
However, in some particular cases where all the releases have the "Packaging" field populated, those releases are ignored since it's missing in the RegExp that checks the returned release. For instance, test the following albums:

http://musicbrainz.org/release-group/9e6...5d99242f61
http://musicbrainz.org/release-group/504...ef84c3ff0e
http://musicbrainz.org/release-group/03a...ca9dfa9bdf

I tested the optional packaging xml tag in the RegExps and then the scraper started getting those releases correctly.
find quote
mikebzh44 Offline
Posting Freak
Posts: 1,710
Joined: Nov 2011
Reputation: 47
Location: Nantes - France
Post: #305
Hi.

Scaning for Eminem album "The Marshall Mathers LP 2" return nothing and you enter an endless loop because XBMC ask you for album name and artist name (until you brake the loop by pressing ESC key of course).

The album is known in musicbrainz :

https://musicbrainz.org/release-group/b1...edca6b2ad3

And if you rename album to "The Marshall Mathers" :

[Image: Z39hi0n.png]

then the scrap is OK for this album

[Image: BazCnW2.png]

Here is my log file with debug ON :

http://xbmclogs.com/show.php?id=237885

Query for musicbrainz API : http://search.musicbrainz.org/ws/2/relea...ist:Eminem

XML look OK but no results are returned :

Code:
11:27:16 T:10812   DEBUG: scraper: GetAlbumSearchResults returned <results sorted="yes"></results>

When renaming album to "The Marshall Mathers" :

Code:
11:27:23 T:12248   DEBUG: scraper: GetAlbumSearchResults returned <results sorted="yes"><entity><year>XE-2000-T#10</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-435eb30f-f461-4f2a-960c-3dc18ce5a6ca-album.xml">http://musicbrainz.org/ws/2/release/435eb30f-f461-4f2a-960c-3dc18ce5a6ca?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>US-2000-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-9a45c98b-3ef7-3395-9f3f-d4dfd0eacf0f-album.xml">http://musicbrainz.org/ws/2/release/9a45c98b-3ef7-3395-9f3f-d4dfd0eacf0f?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>US-2000-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-976ce3a6-7a00-4273-a83e-df1f908246ff-album.xml">http://musicbrainz.org/ws/2/release/976ce3a6-7a00-4273-a83e-df1f908246ff?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>GB-2001-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-29f9ce15-7171-3495-bc32-705192370204-album.xml">http://musicbrainz.org/ws/2/release/29f9ce15-7171-3495-bc32-705192370204?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>XE-2001-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-d5c10a17-6b59-3903-b46e-ae9522c73c50-album.xml">http://musicbrainz.org/ws/2/release/d5c10a17-6b59-3903-b46e-ae9522c73c50?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>US-2000-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-156a7344-89e1-49fb-b3d3-9525c1ee5a5f-album.xml">http://musicbrainz.org/ws/2/release/156a7344-89e1-49fb-b3d3-9525c1ee5a5f?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>AU-2002-T#23</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-51544aed-52a1-42b9-aff0-9237ac3dd564-album.xml">http://musicbrainz.org/ws/2/release/51544aed-52a1-42b9-aff0-9237ac3dd564?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>XE-2000-T#10</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-435eb30f-f461-4f2a-960c-3dc18ce5a6ca-album.xml">http://musicbrainz.org/ws/2/release/435eb30f-f461-4f2a-960c-3dc18ce5a6ca?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>US-2000-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-9a45c98b-3ef7-3395-9f3f-d4dfd0eacf0f-album.xml">http://musicbrainz.org/ws/2/release/9a45c98b-3ef7-3395-9f3f-d4dfd0eacf0f?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>US-2000-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-976ce3a6-7a00-4273-a83e-df1f908246ff-album.xml">http://musicbrainz.org/ws/2/release/976ce3a6-7a00-4273-a83e-df1f908246ff?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>GB-2001-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-29f9ce15-7171-3495-bc32-705192370204-album.xml">http://musicbrainz.org/ws/2/release/29f9ce15-7171-3495-bc32-705192370204?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>XE-2001-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-d5c10a17-6b59-3903-b46e-ae9522c73c50-album.xml">http://musicbrainz.org/ws/2/release/d5c10a17-6b59-3903-b46e-ae9522c73c50?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>US-2000-T#18</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-156a7344-89e1-49fb-b3d3-9525c1ee5a5f-album.xml">http://musicbrainz.org/ws/2/release/156a7344-89e1-49fb-b3d3-9525c1ee5a5f?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>AU-2002-T#23</year><artist>Eminem</artist><title>The Marshall Mathers LP</title><url cache="mb-51544aed-52a1-42b9-aff0-9237ac3dd564-album.xml">http://musicbrainz.org/ws/2/release/51544aed-52a1-42b9-aff0-9237ac3dd564?inc=recordings+release-groups+artists+labels+ratings</url></entity></results>

So, what's wrong with "The Marshall Mathers LP 2" album ?

Thanks.

Sorry for my english, but, you know, I'm French so ...

find quote
VelvetKevorkian Online
Senior Member
Posts: 257
Joined: May 2006
Reputation: 0
Location: Germany
Post: #306
The endless loop thingy you described also happens with artists who have special charcaters, like "Accu§er".

Frodo behaviour: no automatic result, manually type in "Accu$er" - artist gets scraped.
Gotham behaviour: no result at all. No possibility of entering manually.



Edit: Gotham behaviour was sth different.
But i know the "endless loop" when searching for albums etc it cannot find.
(This post was last modified: 2014-06-30 18:44 by VelvetKevorkian.)
find quote
zag Offline
Team-XBMC Member
Posts: 1,668
Joined: Oct 2007
Reputation: 19
Location: UK
Post: #307
If you remove the entire artist or album name and click ok you can break the "endless" loop without having to hit escape.

But you are right this is a regression from Frodo.

I'm not sure its the scraper that causes it though, will check.

HTPC - XBMC Gotham, OpenELEC, Harmony Smart Remote, Intel Haswell NUC, 40gb intel SSD, Core i3, 4gb RAM
Storage - 2 x qnap 8tb 419p+ NAS
Display LG 46" LCD + Casio Bulbless projector [PICS]
[Image: widget]
find quote
mikebzh44 Offline
Posting Freak
Posts: 1,710
Joined: Nov 2011
Reputation: 47
Location: Nantes - France
Post: #308
This PR will fix my issue with Eminem - The Marshall Mathers LP 2 :

https://github.com/xbmc/xbmc/pull/4960

Thanks.

Sorry for my english, but, you know, I'm French so ...

find quote
Scooby-2 Offline
Junior Member
Posts: 6
Joined: Dec 2012
Reputation: 0
Post: #309
Please excuse my ignorance but I am totally new to XBMC. I am running it on a Raspbery Pi to play CDs stored on a NAS server to my (HDMI connected) TV. The problem I have is that I cannot understand the instructions from here to scan the music into the library, so I do not have any album art for any of my CDs. The music plays fine, and I'm really impressed with XBMC. However, to enable album art, right at the beginning it says "...make sure you are not in 'Library Mode' (checkmark on the left)" but I guess I must be in library mode and I can't figure out what it is or how to get out of it. I cannot get the menu that's displayed on the help pages (where it says "Scan item to library") to display. This is where I need to get, this is a link to the image on the instructions:

[Image: thumb.php?f=XBMC12_Scan_Music_Library.jp...;amp;w=800]

How do I get to this screen? I have also checked "Update library on startup" in settings, but I am presuming that the library list is empty so there is nothing to download.

The universal scraper is enabled, btw.

Thanks in advance.

EDIT: Here's what I can see when I go to play the MP3 files. (They are all ID3 tagged, too.)

[Image: IMG_1259croppedresized_zpsc8aca069.jpg]
(This post was last modified: 2014-07-01 22:13 by Scooby-2.)
find quote
VelvetKevorkian Online
Senior Member
Posts: 257
Joined: May 2006
Reputation: 0
Location: Germany
Post: #310
You're in file mode. Click left to go to library mode.

A good noobi guide for XBMC can be found here:
http://wiki.xbmc.org/index.php?title=XBM...tart_Guide
find quote
Scooby-2 Offline
Junior Member
Posts: 6
Joined: Dec 2012
Reputation: 0
Post: #311
@VelvetKevorkian - thanks for the response. I can now get in and out of Library Mode and I have added all my CDs to the Library. I do still have a problem with a total lack of album art but I see from the net that this is usual when files stored on a NAS server. I'm going to confirm this by copying an album to the xbmc itself and see if it gets scraped when it's local, but I need to do this when wifey's not watching the TV!
find quote
bleakassassin Offline
Junior Member
Posts: 11
Joined: Jul 2014
Reputation: 0
Post: #312
I've personally noticed that using the option "Query Info for All Albums" doesn't do anything to my library whenever I use it. I always have to (re)load album information for each album individually if I want to get information on it. Does anyone else have this problem?

Apologies if it isn't related to this scraper.

EDIT: Scratch that. It seems using the Query option does work...but only for a few albums at a time. There are tons of errors in the debug log I produced when I last ran the option.

Debug log
(This post was last modified: 2014-07-12 09:57 by bleakassassin.)
find quote
n1md4 Offline
Fan
Posts: 397
Joined: Nov 2012
Reputation: 8
Post: #313
Hi,

OE 4

I've a few albums that aren't scraping. I'll try 1 at a time - that way I'll may be able eliminate albums with similar issues. Here is the first:-

http://xbmclogs.com/show.php?id=250958

HTPC XBMC Gotham OpenELEC, NYXBoard, Athlon X2, OCZ SSD, 4G RAM
Storage 4x 2TB Green HDD BTRFS RAID1, Athlon X2, OCZ SSD, 4G RAM
find quote
n1md4 Offline
Fan
Posts: 397
Joined: Nov 2012
Reputation: 8
Post: #314
I was hoping for a reply before posting again, but alas ... if I'm ommitting something worth posting let me know!

Here's another example, with similar results, that is, the scrape request hits musicbrainz API, and manually checking the URL, I can confirm it returns a result. What I don't understand is why XBMC doesn't update.

So, another example, if it helps.

The single Free by Estelle.

MB: http://musicbrainz.org/release-group/3c5...7f/details
audioDB entry: http://theaudiodb.com/album/2215036
XBMC log: http://xbmclogs.com/show.php?id=252056

HTPC XBMC Gotham OpenELEC, NYXBoard, Athlon X2, OCZ SSD, 4G RAM
Storage 4x 2TB Green HDD BTRFS RAID1, Athlon X2, OCZ SSD, 4G RAM
find quote
n1md4 Offline
Fan
Posts: 397
Joined: Nov 2012
Reputation: 8
Post: #315
No ideas what so ever?

HTPC XBMC Gotham OpenELEC, NYXBoard, Athlon X2, OCZ SSD, 4G RAM
Storage 4x 2TB Green HDD BTRFS RAID1, Athlon X2, OCZ SSD, 4G RAM
find quote
Post Reply