New Scraper idea - thoughts?

I'm not sure if anyone has been talking about this or not, or if there is even any interest in such a scraper but I really like the idea of scraping "similar films" or "You may also like this". I noticed that IMDB has this feature now on there site (may have been there for a while, I just noticed it) and sites like Netflix and Blockbuster have always had features like that. I'm just throwing out a suggestion as I have no real programming experience and probably couldn't develop such a scraper but I just wanted to see if anyone was interested in this idea. My vision of it would be like this, only with better graphics of course.



Is there anything like this currently available?

mmm.... In theory, if the site being scraped has the information (i.e., IMDB), obtaining it would not be very difficult, the problem is that it should then be stored in the database and there is no place there prepared to this, it would require a new version of the database.

IMHO, the XBMC database has some shortcomings (although some work is being done to improve it), maybe some kind of standarized extensions could be designed so certain addons could write data without breaking anything else; for example, an addon for what you are proposing (then the skins should be capable of detecting and supporting it, but the skins that do not support it would simply work without the functionality).

There are huge opportunities there, some scrappers could create their own extensions (for example, the host for docummentaries or the guests in talk shows), the skins could store how to display a certain path (so if you change skin and then come back, your settings for each skin will be kept), and a long etc.

But I'm afraid all these is very low level functionality, it probably will affect an enormous part of XBMC code and so it qould require lots of dev's time&work,

My GSoC project, clean scraper API http://www.google-melange.com/gsoc/proje...skog/23002 will most likely allow for this. My hope is that it will allow the scraper creators to give essentially any information they want (beyond what is suggested) and it will be stored so that skinners may use it. However my project will not make XBMC database part work, which would be essential, but it would be one step in this direction.

I'm hopeful that this sort of thing could eventually be implemented, just thinking of all the cool possibilities. Like when it pulls up say 6 movies that are similar and 4 of them are in your library - you can watch them right away and if the other 2 aren't in your library there could be a couchpotato implementation that you could queue up. I just really like the idea of finishing a good movie and seeing suggestions to other similar movies - the concept being that you can do everything straight from XBMC instead of going to IMDB and searching. I'm constantly watching movies and searching for movies that I haven't seen so something like this would be amazing.

(2012-05-31 16:48)tman12 Wrote:  I'm not sure if anyone has been talking about this or not, or if there is even any interest in such a scraper but I really like the idea of scraping "similar films" or "You may also like this".

Yes I would love to have a "similar movies", "similar TV-series", and "similar artists/songs" for music feature in XBMC by default

Checkout requests http://forum.xbmc.org/showthread.php?tid=130917 and http://forum.xbmc.org/showthread.php?tid=38607

(2012-05-31 16:48)tman12 Wrote:  I noticed that IMDB has this feature now on there site (may have been there for a while, I just noticed it) and sites like Netflix and Blockbuster have always had features like that.

Other recommendation system sites / sources to scrape for movies is SynopsiTV / Synopsi.TV and TheMovieDB.org

Last.fm and MusicBrainz have the same type of "similar" information for music artists and songs