+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Kodi Application (https://forum.kodi.tv/forumdisplay.php?fid=93)
+---- Forum: RetroPlayer Development (https://forum.kodi.tv/forumdisplay.php?fid=194)
+---- Thread: Games Library (Game & Emulator database + launcher) development project, help wanted! (/showthread.php?tid=40715)
- VictorV - 2009-11-20
Nicezia Wrote:GiantBomb Game Scraper
A simple implementation to begin with
no fanart yet, gets several box covers, and i can't yet figure out how/or why it's not grabbing the description (at least in scraperXML) as i haven't tried it in your GamesLibrary branch (computer currently won't run XBMC of any flavor, since my nVidia card burnout)
Still got that scraper? I would love to have a scraper to test with
- tmpdir - 2009-11-20
thanks greatant1337 Missed that thread completly in my search (will read it this weekend). Without reading it I'm convinced the requirements are of the kind 'moving targets'.
- tmpdir - 2009-11-20
rickx Wrote:I'm not exactly an svn guru. If I check out the games-branch, what version (revision) will it be? Is it possible to merge it with the current rev.? If so, could somebody drop the code for doing it? (Apart from probably having to solve some issues/conflicts manually...)
svn merge is quite easy to use, just use google. Expect some minor merge issues, some of the depencendies have been updated recently.
- rickx - 2009-11-20
tmpdir Wrote:svn merge is quite easy to use, just use google. Expect some minor merge issues, some of the depencendies have been updated recently.
Yes, I've heard google before...I also had tried already, but if someone could kindly comfirm it is right, I could spare some precious time...
I checked out the branch in one folder, the latest in another.
Then(?) I went inside the latest folder and used
Code:
svn merge ../gamebranchBut maybe it should be the other way round. Or adding parameters...
- tmpdir - 2009-11-21
Next time you might want to mention what you've already tried... to many people ask before getting their hands dirty.
It's been a while, but its something like:
svn merge \
https://svn/trunk/equal_directorie_name/src \
https://svn/branches/equal_directorie_name/src \
.
The dot is your current directorie where the merge should arrive. After checking the results etc, you can commit . to your working folder.
Edit:
- the 'equal_directorie_name' folders must be the same.
- works the same with local folders
- the . points to you current dir.
- Mityama - 2009-12-01
Hello everybody,
I have been thinking about creating a game database as an alternative to something like allgame. But in contrast to those, it would be XBMC specific and contain exactly the info and media in exactly the form needed.
Furthermore.... I am not a scraper expert, but how does the following sound as a way to grab game information with a very high accuracy rate:
- XBMC calculates the MD5 hash of a game ROM, then
- sends it to the site via a HTTP request like "http://[site]/?md5=5443508gfbfg98zh",
- where a small script looks up the needed data in the database, and
- passes it back to XBMC in some form (XML?)
I would need to include all known hashes in the database, but I think this can be automated with some Good Sets and a little PHP script. I could also implement a search for the game title if a ROM is unknown.
But before I start getting more into this, I would like you guys' opinions about this first. Is it worth the effort? Would it help the game library project?
- VictorV - 2009-12-02
Mityama Wrote:Hello everybody,It's a shitload of work, but it would be awesome if you did.
I have been thinking about creating a game database as an alternative to something like allgame. But in contrast to those, it would be XBMC specific and contain exactly the info and media in exactly the form needed.
Mityama Wrote:Furthermore.... I am not a scraper expert, but how does the following sound as a way to grab game information with a very high accuracy rate:I would recommend crc since that's what is commonly used in various communities
- XBMC calculates the MD5 hash of a game ROM, then...
Mityama Wrote:- sends it to the site via a HTTP request like "http://[site]/?md5=5443508gfbfg98zh",you might also want to add a fuzzy search based on file name and platform in case there's no known crc.
Mityama Wrote:- where a small script looks up the needed data in the database, andIt's a lot of dat-files around containing crc¨'s for rom collections
- passes it back to XBMC in some form (XML?)
I would need to include all known hashes in the database, but I think this can be automated with some Good Sets and a little PHP script.
Mityama Wrote:I could also implement a search for the game title if a ROM is unknown.I also think that high quality data is the key.
But before I start getting more into this, I would like you guys' opinions about this first. Is it worth the effort? Would it help the game library project?
I've been playing around with the game library and have made some progress there. You probably want to start early designing a descent database, and you could see what attributes you might want (Should probably be discussed before settled). There's also a need for a good structure handling differences in platforms and games on cross of platforms (cover art and screen shots etc. will vary, but not the plot)
- rcoops - 2009-12-02
I would be more then happy to help out with this effort of setting up a good as complete as possible games database.
I think that hashing is definitely an option but I fear that there are a lot of versions of roms for many games. Looking at the thousands roms I happen to have sitting on a disk somewhere there are often 2 to 5 different versions of the same game sometimes the language changes sometimes the way the rom has been plucked from its original storage etc... a look up based on platform and game title would definitely be needed, but also a series of names for the same rom, different countries often have different names for the same rom.
I think the first thing would have to be to agree on a basic set of data required for each rom, expanding on this later on is a lot easier to do simply add more tags in the XML and mark them as optional for people using the XML. That way you do not have to add the information for all roms but only for those where it is known.
@Mityama if you are interested in help with this massive undertaking please don't hesitate to drop me a PM. Despite being short on time most of the time I would gladly spend some of my free time on a project like this.
- JustinAiken - 2009-12-02
All I'd really like to see for games scraping is
Title (and orig title)
fanart/box
Genre
Year
Company
- rcoops - 2009-12-02
I would add to that:
- Platform
- Developing studio
- Manual
Basically I would expect the site to provide a complete picture including as much information as one could want for every single game. Maybe even adding a walk trough and a cheat(s) part for completeness sake. I know that for some games it helped me a lot to finally after all these years figure out how to get past this or that part of the game...
- mngrl1 - 2009-12-03
This all sounds Awesome guys am very happy to see people still interested in it. Just one thing scraping manuals and cheats canary and all that sounnds really good but can't we just start simple. Think there would actually be more likely for this project to see the light of day rather then putting way to much in and never finishing it
- mngrl1 - 2009-12-03
Lol still getting used to typing on this Iphone canary lol was ment to be fanart
- VictorV - 2009-12-03
This is attributes I'm working with now:
- Title
- Plot outline
- Tag line
- max players
- votes
- Rating
- Region
- Artworks
- Alt. Artworks
- Ext. Artworks
- Action shot
- Intro. shot
- Year
- Thumbnail
- Perspective
- Genre
- Developer
- Publisher
- Video Preview
- Fan art
- Platform
The artworks use some general naming since it could be box art for console games, but cabinet for arcade games.
Both action and intro shots are commonly used, so I figured to stick to that.
Genre would be action, adventure etc, while perspective could be side-scroller
I think manuals are more of an in-game feature, but it could be considered on a later stage.
- rcoops - 2009-12-03
Sounds good to me.
As for the manual, walk trough and cheats I would just add them as optional extra's only there where available and rely on visitors for the most part to populate these.
The only thing that I worry about a little is that certain game studio's might have an issue with box/card/arcade art being made available. Nintendo has a team of lawyers looking into the telephones that are able to run emulators in an attempt to charge for it or ban them completely as it considers these games it's intellectual property and any use even 10 or more years after they stopped selling them should be paid for. They might react in the same way to box/card/arcade art (their design after all).
It might be a good idea to contact them and try and get a ok from them. Usually big companies like that will allow these types of thing when asked, they only crackdown on it when they think there is real money to be made from it. Once you have a written approval any legal action they might feel they need to take later on will be severely hampered by such correspondence so if they do approve don't loose that email.
- rausch101 - 2009-12-03
I always enjoy seeing interest/work in this thread. Nice job guys. And remember, since there's currently nothing implemented, don't get bogged down by extras before you get the core stuff done
rcoops Wrote:Sounds good to me.
The only thing that I worry about a little is that certain game studio's might have an issue with box/card/arcade art being made available.
If Games posters are at all analogous to TV Shows, Movies or Music posters, then this isn't a problem at all. I imagine the websites from which he will be scraping will have dealt with all of that legal stuff anyway, if there was any.