Posts: 12,706
Joined: Nov 2003
Reputation:
129
spiff
Team-Kodi Member
Posts: 12,706
that would be neat, and some system to cross reference several scrapers in order to make matches more reliable
Posts: 4
Joined: Jun 2008
Reputation:
0
2008-12-30, 12:33
(This post was last modified: 2008-12-30, 12:40 by kriziz.)
AllMusic.com isn't updated on a regular basis and doesn't list many relatively unknown artists. I'd expect a boost in search results from almost any new scraper, if its last.fm or discogs.. These two sites are both as good, provide a decent API etc.. One might be better than the other when it comes to specific musical nices. Most people are better off to use their favorite scraper for regular searches. We might want to make it a bit easier to switch scrapers, from the Album/Movie info screen or the manual search screen maybe ?
Anyway, in meantime I did some testing on a discogs.com scraper. This scraper script is just a few lines, it has CreateAlbumSearchUrl and GetAlbumSearchResults nodes and thats about it. Where I get stuck is testing the thing, XBMC simply crashes if I use it (no messages in xbmc.log).
The 'Scrap' test tool looks promising, however it is built to handle video scrapers, not music. Tried to fix that as well, but got caught up in C++ syntax :S Scraper development would be much easier if anyone could fix that Scrap tool. Doesn't look like an easy job, though ..
Am I missing a development tool here, should I be debugging from Visual C++ etc. ?
Posts: 12,706
Joined: Nov 2003
Reputation:
129
spiff
Team-Kodi Member
Posts: 12,706
scrap is broken as we lost the source (blame donj for bad committing practices).
the way i do scrapers is using a regexp tool + printf. it works just fine for me, but an updated scrap tool would be invaluable for sure
Posts: 12,706
Joined: Nov 2003
Reputation:
129
spiff
Team-Kodi Member
Posts: 12,706
basic scraper added to svn
Posts: 12,706
Joined: Nov 2003
Reputation:
129
spiff
Team-Kodi Member
Posts: 12,706
not even a 'it does not work you silly sod'?
Posts: 12,706
Joined: Nov 2003
Reputation:
129
spiff
Team-Kodi Member
Posts: 12,706
shouldnt require anything liek that no.
last.fm offers very little info. just artist, album name, year, releasedate, review and track listing.
even less for artists, although some of the biography's are good.
you did not get any info at all?
Posts: 484
Joined: Sep 2004
Reputation:
0
Would you be able to scrape each tracks 'listeners' and apply it to the 'ratings' field of the database?
each album might need to be calculated relative to itself
Posts: 26
Joined: Sep 2004
Reputation:
0
2009-02-17, 19:55
Hey spiff,
i tried out your last.fm scraper and must say it works brilliant. The only downside are the crappy artist/album pics from last.fm but that is not your fault.
It would be nice if some kind of a language selection would be possible. Perhaps with a fallback to english if no information for the selected language is availabe.
Thanks for your great work
TheNME123