script.module.metautils dev - Printable Version
+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Python Add-on Development (/forumdisplay.php?fid=26)
+--- Thread: script.module.metautils dev (/showthread.php?tid=109725)
[dev] script.module.metahandler - Eldorado - 2011-09-09 03:48
I thought I would get the ball rolling on getting the next piece in the proposed Video Falcon project - common meta data script
I've put the initial version on git - https://github.com/Eldorados/xbmc-metautils
This version was pulled from the master branch of Icefilms, I've basically put everything into it's own script folder and set it up to stand on it's own
* addon calls this module with a episode/movie imdb id or title
* module looks in its database for metadata for the required episode/movie
* if not found it scrapes a site for the required metadata, adds it to the database and returns it
* if it is found, it simply returns it from the database
* search by name and return a list of possible matches
When searching by just movie/tv show name I recommend to try and pass in as clean a name as possible, strip anything that is not apart of the actual name eg. many sites display 'The Hangover (2009)', you need to strip (2009) from the name and pass the year in separately
If you wish to download the covers to the cache folders, in prep to release a meta data zip pack to users, specify the preparezip=True option
Search for movie:
Search for tv series:
Search for tv show season covers:
By this point you *should* have the imdb id of the tv show, if you don't have it then no results will be returned
get_seasons() returns a dictionary for each season found in order
Search for tv show episode:
Update watched status:
Will update status in DB from 6 to 7 or reverse depending on what initial value is, 6 = unwatched, 7=watched
Search for a movie by name and return a list of possible matches:
Meta Data being collected
- LOTS of code cleanup/optimization
- add more meta - director, writers, cast DONE
- fleshing out methods and how they are called
- fix unicode problems, possibly integrating with t0mm0.common DONE
- metacontainers needs attention
- create a metacontainer zip file to optionally download instead of creating a blank DB ala icefilms
All welcome who are wanting to help!
- t0mm0 - 2011-09-09 18:33
Eldorado Wrote:I thought I would get the ball rolling on getting the next piece in the proposed Video Falcon project - common meta data script
is there a specification somewhere for what this bit is meant to do? i'm not very familiar with the metadata stuff so would be interested in what function this bit fills!
- rogerthis - 2011-09-09 19:50
Eldorado, could you get an example of how to execute this in an addon. I had a quick look at the code and I don't know where to start.
- Eldorado - 2011-09-09 20:10
Hi guys, I haven't dug into it yet to do up an example.. though the master branch of Icefilms currently uses it, so maybe a good place to look if you want to jump in?
Basically it needs work before it can be used in any sort of way outside of Icefilms as I said it has quite a bit of specific coding done
t0mm0, this was another item on the Video Falcon list:
- t0mm0 - 2011-09-09 22:37
Eldorado Wrote:t0mm0, this was another item on the Video Falcon list:
so am i right in thinking what it needs to do is...
is it possible to use the existing xbmc scraper modules rather than having separately maintained ones? (just asking the question - i know nothing about metadata in xbmc)
i assume the intention is to maintain a central database so that if a movie is added from one addon its metadata will be available from all others using the module?
seems what is really needed is to be able to add stuff to the main xbmc library. there is also the hack that is doing the rounds at the moment with creating loads of strm files which is trying to solve the same problem i guess?
also there is the mention of building pre-packaged metadata bundles - this sounds like a nightmare to me but maybe there is a particular use?
there should probably be a definition of what metadata is required. does it include posters/thumbs for example? maybe this would also be a good place to track watched status (especially as it would wok across addons) while we can't do it in xbmc?
(i always find it better to try and define what something is supposed to do before writing code - might save rewriting it too much later. i hope the questions above aren't too silly - as i say i don't know anything about metadata in xbmc )
ps. eldorado you need to add .pyo (and .pyc while you are at it) files to your .gitignore file in this repo!
- slyi - 2011-09-10 18:20
I was also tinkering with meta data updates using asynchronous methods for icefilms see demo on http://dl.dropbox.com/u/6589941/asyncmeta/plugin.video.asyncmeta.zip
I think a generic system should only download whats requested at the time and be text only (no images) as these are better stored online rather filling the limited hd of embedded devices apple tv etc...
I'd be interested in helping on this aswell, can you provide a sample that works with ice films v12?
- Eldorado - 2011-09-11 19:41
t0mm0 Wrote:so am i right in thinking what it needs to do is...
I think you nailed it here, pretty much what I was thinking the main functions should be, I'll add this to my op
t0mm0 Wrote:is it possible to use the existing xbmc scraper modules rather than having separately maintained ones? (just asking the question - i know nothing about metadata in xbmc)
Very good question and one I've been asking myself too!
Hoping someone can jump in with the knowledge to give a yay or nay, as your right it's very redundant and quite a bit of extra work to write and maintain a separate scraper
t0mm0 Wrote:i assume the intention is to maintain a central database so that if a movie is added from one addon its metadata will be available from all others using the module?
All good points, I guess initially I was thinking to basically get this module running on it's own first and keeping all the current functionality that it is performing with Icefilms - pulling in all metadata from plot, genre, cast, thumbnail etc. storing it all in a local cache accessible by any addon, set those as it's initial boundaries then work towards defining what phase 2/enhancements should be.. eg as you said adding to main library, pre-packaged meta containers etc
t0mm0 Wrote:ps. eldorado you need to add .pyo (and .pyc while you are at it) files to your .gitignore file in this repo!
Eeee.. I usually make sure I don't copy those files
- Eldorado - 2011-09-11 19:50
slyi Wrote:I was also tinkering with meta data updates using asynchronous methods for icefilms see demo on http://dl.dropbox.com/u/6589941/asyncmeta/plugin.video.asyncmeta.zip
I'm not sure the user would like a system that has to re-scrape every time you pull up a list of movies, and I'm assuming nor would a site such as TMDB
The Apple TV has I believe 2gig storage space?
Perhaps an option between saving just text vs text & images?
The code I have posted only works with the master branch of Icefilms due to the number of changes, Anarchintosh had said it was 95% complete, don't see any notes on what is left to do..
If you need a v12 version you can simply pull it from the current addon folder, will need to modify to remove all the icefilms specific logic
- anarchintosh - 2011-09-14 18:21
asynchronous updates are only realistic if you only use low quality thumbnail cover art instead of high quality cover art, which is a bit of a sad tradeoff.
- Eldorado - 2011-09-24 00:03
I've done some small updates
- removed all (that I could find) icefilms specific coding, which at quick glance appeared to be scraping the icefilms site for metadata if a IMDB id did not exist, possibly something like this might be useful for other sites, something to keep in mind for updates
- small changes to use getAddonInfo('path') and sqlite3
Below is a quick example on how to scrape for a movie or tv show, the metadata will be stored in a sql db in the addon_data folder
Still quite a bit left to do which mainly consists of code cleanup
Also looking for info on the possibility of using the existing TMDB and TVDB scrapers so that a second set does not need to be maintained
edit - found my answer, no can do currently : http://forum.xbmc.org/showthread.php?tid=89245&highlight=scraper