MusicBrainz & extending the MusicScanner/Scraper

Hi,

Noticed MusicBrainz support is sort of implemented with the MB tags cached in CSong. I was taking a look into writing an MB scraper and I noticed that the scrapers only get the Artist/Album as strings. I took a look at the code behind the MusicScanner & MusicScraper and wondered if you would be open to discussion & patches to the music scraper?

I looked at a couple of approaches to propogate MBIDs to the scrapers in the code, but my thought finally was to do the following - open to feedback:
1) I would propose to add the fields MusicBrainzArtistID -> 'artist'/CArtist, MusicBrainzAlbumID & MusicBrainzAlbumArtistID -> 'album'/CAlbum in MusicDatabase->AddSong. While I was writing this I realised this has a secondary advantage - later we can disambiguate similar named Artists/Albums etc using the strArtist & MBArtistID as the key on Artist. But in the first instance the proposal is just to propogate these fields to the Artist/Album to keep it simple. There would be issues around similar named artists but I think those same issues exist today so we're not breaking anything existing.

2) For MusicInfoScanner and MusicDatabase:
My proposal is to pass a CAlbum, CArtist, etc to DownloadAlbumInfo/DownloadArtistInfo and downstream functions instead of fixed strings strAlbum and strArtist. This way any members of the CAlbum & CArtist structs can be passed to scrapers. Later if we extend those structs it is easier to get that data to the scrapers (no refactoring the function params each time). For those source functions calling e.g. DownloadArtistInfo with only a string we'd first create a CArtist, and populate it with all of the data we have in hand. The scrapers could get more of the data available to us in order to make less amiguous decisions. With MBIDs in CArtist & CAlbum as above, we can now easily pass the scanned MBIDs to the scraper.

I have working patches to do 1) and I'm just working on part 2) of the change as a proof of concept. I notice also the MBID tags on my m4as aren't being scanned in right now, I will check that out too.
Finally this would require additional scrapers that key off MBID not 'Artist Name' etc.

I think all of this would be great for fanart.tv which already seems to key of MBID, artist disambiguation, etc.
Wanted your feedback on whether this would be something useful and considered/reviewed for inclusion?
Thanks in advance for looking.

Sounds like a decent plan to me. I think to be completely useful we'd also need to compute/lookup the mbid for songs that don't have it, as my guess is that 99% plus of users would not have that information available in their tags. But ofcourse, that is something separate.

Sure. I have some ideas around that part as well, but I don't want to get ahead of myself right now. If folks are okay with the approach - let me complete the first part and post a link to the git for review/testing soon.

Best Regards.

I haven't read through your ideas but all I know is that olympia (who also writes and maintains the TVDB, TMDB, IMDB and AllMusic scraper) is currently working on a MusicBrainz scraper (or maybe has already finished) it. So maybe you wanna contact him so you guys don't do similar work twice.

(2012-05-28 09:14)Montellese Wrote:  I haven't read through your ideas but all I know is that olympia (who also writes and maintains the TVDB, TMDB, IMDB and AllMusic scraper) is currently working on a MusicBrainz scraper (or maybe has already finished) it. So maybe you wanna contact him so you guys don't do similar work twice.

Artist scraper is working.
Album part is still being worked on.


Being able to store the MBID in in database table(s) (like artist, album group-release or/and release itself, song, record label) would be very useful IMO.
So you get a big +1 from me

(2012-05-28 09:14)Montellese Wrote:  I haven't read through your ideas but all I know is that olympia (who also writes and maintains the TVDB, TMDB, IMDB and AllMusic scraper) is currently working on a MusicBrainz scraper (or maybe has already finished) it. So maybe you wanna contact him so you guys don't do similar work twice.

released today:
MusicBrainz Album Scraper

metadata.album.musicbrainz.org
metadata.common.musicbrainz.org

I've made the first set of changes to the MusicScanner & DB required to get this going.

They're a little kludgier than I would have preferred as I had a couple of misconceptions around how the scrapers & scanners worked and I'm working round a few things. I think if this works out I could offer to work on some housekeeping in the (music) scanner & scraper code to make it a bit cleaner. I have some ideas to add AcoustID support later but I'm thinking the current code might want some TLC first? One thought is to more fully integrate the NfoReader as a proper scraper into the scraper code at GetScraperForPath instead of in the individual Album/Artist downloaders.

Forked repo is here. Open to feedback.
https://github.com/night199uk/xbmc

Scrapers done.
https://github.com/xbmc/xbmc/pull/1027

I'm chatting offline with olympia about whether he would be interested to write the actual scraper code for this. I will start on MBID parsing for m4a/AACs as my music is mostly m4as with MBID tags.

Regards.

The initial patch I've provided should take the first (primary) artist and album artist.

Looks like in NGS Picard can now write multiple artist and album artist tags for e.g. 'Mark Knopfler & Chet Atkins' using their credit system.
This doesn't fit well with the idArtist & strExtraArtists schema currently in use (unless we do something like strMusicBrainzArtistID/strMusicBrainzExtraArtistIDs).
To me the cleanest solution here would be to be able to make the song/album -> artist table links N:N.

I don't want to tread on the whole schema design conversation as that seems to be a huge thread - I don't want to try and boil the ocean.
My thought right now is to evaluate the work / pitfalls of converting the Album/Song -> Artist table mappings to an N:N mapping, and also importing/supporting things like the Compilation flag on MP3s for identifying Various Artist releases more easily.

My next step is to get the primary MBID data imported from M4As though.

They already are N:N. The strExtraArtists is only there to speed things up for the UI (denormalisation for speed). See the link tables exartistsong/album.