[WIP] Audio-Matic Automated Music Downloader/Organizer

ProphetVX Wrote:I've attempted a similar project before, the biggest issue I found was the lack of sites to easily scrape accurately that actually had a comprehensive and accurate database, unfortunately with the exception of itunes, most sites have limited scope or catalog's, so the difficult part was merging that data across various sites in a useful manner. It seems like you've overcome those issues though, so keep up the good work, I'm very interested to see how this project progresses.

Between http://www.discogs.com and http://www.allmusic.com, most information can be obtained, but it is not always current. I cross-check the information obtained and validate it against any ID3 tag information that can be obtained from existing files. I haven't tried obtaining information from itunes, but I'm actually going to look into that right now since you mentioned it.

ProphetVX Wrote:Based on your screenshots, I'm assuming the application is written in C#? Another option is to port it to java, which there are many obvious syntactical similarities which would create a far easier path to having a hosted solution and platform independence, but that's obviously a fair way down the roadmap. Depending on the complexity of the project there is a plugin for netbeans that I've used with moderate success that can automate the porting of VB or C# to Java.

This project is currently written using VB.NET (2010 version). C# is very similar though... mostly minor syntax differences. I can fairly easily convert back and forth if needed. Converting VB.NET to Java would be a bit more of a challenge though. I'm at around 7,000 lines of code at the moment so that would definitely take some time.

Actually, I think I could write a scraper for itunes without too much difficulty (i'll have to play later)...

It looks like I can send a query on their website using something like this:


This may have some limitations though. I'll analyze it with firebug when I get some time.

Edit:

This is the ajax call structure call that happened when I searched for "Breaking Benjamin". It outputs a nice XML file for scraping...


Edit #2:

Even better! Looks like they have a documented API. Didn't think it would be that easy.

http://www.apple.com/itunesaffiliates/AP...rch2.1.pdf

And since I already scrape from AMG (AllMusic) IDs and iTunes allows you to do a search based on AMG IDs, this is easy as pie to add... but then again it looks like Apple may get their info from AllMusic so not sure how relevant this may be. If nothing else, I can at least pull out the iTunes album links (like this: http://itunes.apple.com/us/album/dear-agony/id332288664) and albumart Here's the sample query for the same artist though:


What would be really nice is if I can extract information from existing iTunes Library XML files on a users PC if they use iTunes to manage their music collection.

Holy-shit-moly! This has come on a bundle since I first heard about the concept. Great work CC. I think the same as the rest though, should be web interfaced where possible, but we are all willing to wait - we have waited this long! A tad longer wont be too difficult!

I'm just a little concerned about the strain this could put on the NZB sites. Keep that in mind! Music will be sought after much more than TV & Movies and the gazillions of tracks available online will take a whole load of finding/indexing!

DejaVu77 Wrote:I'm just a little concerned about the strain this could put on the NZB sites. Keep that in mind! Music will be sought after much more than TV & Movies and the gazillions of tracks available online will take a whole load of finding/indexing!

Thanks for the support. Interesting thought though. I will have a built-in rate limiter for search requests, but if enough people are using this that could still become a problem. Nothing else I can think of to help overcome that though other than falling back on the built-in torrent support.

With enough providers supported the load should be spread out pretty well. I'm sure Sick Beard has caused an increase in load too. But people have been using RSS feeds and other scripts before and still do, so I wouldn't expect it to be a problem.

Not knowing anything about coding, it might be best if you just continue to work in VB.NET. After you get the code cleaned up and working the way you like you could then release the code and the community could help translate it to python.

Can't wait to use this. No matter what platform its on. I can wait for linux support.

DejaVu77 Wrote:Holy-shit-moly! This has come on a bundle since I first heard about the concept. Great work CC. I think the same as the rest though, should be web interfaced where possible, but we are all willing to wait - we have waited this long! A tad longer wont be too difficult!

I'm just a little concerned about the strain this could put on the NZB sites. Keep that in mind! Music will be sought after much more than TV & Movies and the gazillions of tracks available online will take a whole load of finding/indexing!

Initial searching would cause a spike in traffic but I highly doubt music would have more traffic than TV Shows. I certainly don't have a wishlist of 15-20 albums per week waiting to download. Nor would I do a back catalog of 200+ albums in one hit like you'd do with a TV Show. You can't download single tracks on newsbins, so unless you're an absolute audiophile who needs to download more than 2-3 albums a day you're unlikely to generate more traffic than videos would. In saying that all programs such as this should have rate limiting built-in. Scraping and having automated downloads is always a good excuse to get banned.

This looks fantastic, any plans for a Mac or Python based version?

Not sure if it's been mentioned, but I've used tagging software that utilizes the MusicBrainz database with great success. I'm not sure if they offer an API-like method of querying, but it might be worth a shot in looking to see if it's useful.

http://musicbrainz.org/

Edit: looks like they do indeed http://musicbrainz.org/doc/Developer_Resources

Been waiting for something like this for ages! Cannot wait until release, any kind of timeframe?

rockstrongo Wrote:http://musicbrainz.org/doc/Developer_Resources

At one time I looked into this and planned to support it, but I'd forgotten all about it. I don't think there is a need for it in the initial release, but I will definitely add support for them in the future.

dannycorker Wrote:Been waiting for something like this for ages! Cannot wait until release, any kind of timeframe?

I wish I could give an exact date, but this is largely dependent on my work schedule and family life. I have two full-time jobs (own a computer business and work full-time as an IT Specialist for another company), a wife (need I say more), and 3 kids (4th is on the way... due any day now). That being said, I still usually find time to work on this at least a few times a week.