[Gamester17]

I'm started working on a Last.FM Scraper

Is it true that LastFM limit your query to one per second? If so then lookup with LastFM would take a very long time if you have a lot of songs.

All web services here are for non-commercial use only under the Creative Commons Attribution-NonCommercial-ShareAlike License. If you want to use these services in a commercial context you must contact partners at last dot fm first to get a seperate license.

You can query last.fm 1 per second, averaged over an hour. Which is 3600 requests per hour. Normally your only making one request every 3 to 6 minutes (song length).

The xml feeds and other info @ http://www.audioscrobbler.net/data/webservices/

Though the cool thing about LastFM is that you could also grab additional information such as information about and picture of the artist, though for that you might have to request XBMC's skinning-engine to be extended for first, so tthat is off-topic for this thread.

[spiff]

seeing we never grab info for songs online (apart from the track listing of an album), that really is not an issue

[Kyusaku]

I would love a Last.fm scraper, Allmusic keeps scraping the wrong artist info. Because actually a good amount of my collection are Japanese indie bands.(most of which have Last.fm listings. )

[Gamester17]

LastFM scrapers in development if progress, see here if you like to help:
http://forum.xbmc.org/showthread.php?tid=38379
and
http://forum.xbmc.org/showthread.php?tid=39028

[azido]

There's also an API for the site, maybe worth looking at:

http://www.discogs.com/help/api

TheUni

it would be extremely cool to use discogs as a scraping source. as far as i can see they've have a standarized output and hellofalot of informations.

man i wish i could be skilled enough to write a scraper for it. allmusic keeps adding the complete wrong informations to my albums since it may choose a "hit" by the first matching target it sees. at least when doing autoupdate for the music database.

[DuMbGuM]

it would be extremely cool to use discogs as a scraping source. as far as i can see they've have a standarized output and hellofalot of informations.

man i wish i could be skilled enough to write a scraper for it. allmusic keeps adding the complete wrong informations to my albums since it may choose a "hit" by the first matching target it sees. at least when doing autoupdate for the music database.

Yeah, and I was able to tag my entire collection with discogs, it has everything. I'll start using XBMC again if theres support for a decent scraper.

[Gamester17]

How about a music scraper for MusicBrainz.org?

MusicBrainz hold a nice (and growing) community music metadata database that attempts to create a comprehensive music information site of ID3 tags and works with digital audio the way CDDB/freedb works with CDDA so it should be able to make a standard XBMC music scraper for musicbrainz.org

MusicBrainz is free site and they provide free open source API to access the db. It will be great to have MusicBrainz scraper in XBMC.

The mea data collected by the MusicBrainz project is made available to the public under open licenses. Some of the data is available under the Public Domain, and some under the Creative Commons Attribution-NonCommercial-ShareAlike license

http://musicbrainz.org
http://en.wikipedia.org/wiki/MusicBrainz

Note! I am not here talking about their acoustic fingerprint algorithm library which is suggested here:
http://forum.xbmc.org/showthread.php?tid=37230
I am in this thread only suggesting a basic music scraper for MusicBrainz's online API
http://musicbrainz.org/doc/Development
http://musicbrainz.org/doc/Database

[mbeyer]

How about a music scraper for MusicBrainz.org?

MusicBrainz hold a nice (and growing) community music metadata database that attempts to create a comprehensive music information site of ID3 tags and works with digital audio the way CDDB/freedb works with CDDA so it should be able to make a standard XBMC music scraper for musicbrainz.org

MusicBrainz is free site and they provide free open source API to access the db. It will be great to have MusicBrainz scraper in XBMC.

The mea data collected by the MusicBrainz project is made available to the public under open licenses. Some of the data is available under the Public Domain, and some under the Creative Commons Attribution-NonCommercial-ShareAlike license

*bump*

I really want to bring this back up.
MusikBrainz is a quite complete collection. Allmusic can't compare to that in any way.

[Spectre5]

Any progress on a musicbrainz scraper (for albums)?

[isamu.dragon]

+1 for a musicbrainz scraper

[zag]

Scraping musicbrainz has problems with performance from experience, its not reliable.

Sounds like an open, community driven, music database with a standard API would help. Covers, fanart, track-listings, ratings, comments and lyrics would be nice....

:p

[Martijn]

Scraping musicbrainz has problems with performance from experience, its not reliable.

Sounds like an open, community driven, music database with a standard API would help. Covers, fanart, track-listings, ratings, comments and lyrics would be nice....

:p

MusicBrainz database could be mirrored...
API is already available..
fanart.tv is closely linked to to MusicBrainz concerning MBID

[zag]

Just on that subject, do you know what ID's XBMC stores or would be interested in concerning audio scraping?

I was thinking allmusicID, MusicBrainzID

Any other online site that might offer some nice content? Does fanart.tv have a unique ID for artists?

[Martijn]

Just on that subject, do you know what ID's XBMC stores or would be interested in concerning audio scraping?

I was thinking allmusicID, MusicBrainzID

Any other online site that might offer some nice content? Does fanart.tv have a unique ID for artists?

No ID is stored in XBMC. All is scraped using artist name/album title.

If you use an sql browser you can see that there are some ID fields in the song table IIRC however these aren't used.

Already asked olympia if he could look into utilizing those. That will save tons of headaches and speed up searches.


fanart.tv only uses MBID for artists/albums.

MusicBrainz has links to to several sites for each artists (which are ofcourse added by users).
There are also avaiable through the API.

So if you have the MBID and the artist has a link to allmusic page you will also have that ID.
IMO MusicBrainz could be the central place for your artists/albums. Ofcourse this will require user contributions like adding links and info.

If speed will become an issue we can look into mirroring MusicBrainz database to a server managed by us for example.
We already thought about that for internal use of fanart.tv to speed up the site and perhaps only let a couple of add-ons use it like cddart manager

[olympia]

http://forum.xbmc.org/showthread.php?tid=132592