XBMC Community Forum
Icefilms (Icefilms.info) Addon Development Thread - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Python Add-on Development (/forumdisplay.php?fid=26)
+--- Thread: Icefilms (Icefilms.info) Addon Development Thread (/showthread.php?tid=90315)



- rvas18 - 2011-12-04 03:03

Eldorado, have you considered adding your git to your sig? It drives me nuts digging through the thread everytime i want to review changes. I guess im to lazy to bookmark


- Eldorado - 2011-12-04 18:42

zpanderson Wrote:It is an interesting question we've got here... I was just messing around and set it to wait until the file is 100mb and show the progress dialog. I'm not sure how to determine what the minimum would be though...

[EDIT] I thought about it some more and decided that we can get the file size from megaupload page and then have a setting to say what % of the file we want cached before it starts playing (updated code snippet as well)

Code:
source=GetURL(url)
        match=re.compile('<strong>File size:</strong> (.+?) MB<br />').findall(source)
        fileSize = float(match[0])
        minSize = (fileSize * int(selfAddon.getSetting('download-percent-before-playing')) / float(100)
        #minSize = (fileSize * int(40)) / float(100)
        print 'This is the minSize: ' + str(minSize) + ' and this is the fileSize: ' + str(fileSize)
        dProgress = xbmcgui.DialogProgress()
        dProgress.create('Downloading', vidname, '     will start playing as soon', '      as it reaches the minimum size')
        while 1:
            if os.path.exists(mypath):
                downloadedPercent = int((os.path.getsize(mypath) / (float(minSize)*1000000)) * 100)
                dProgress.update(downloadedPercent)
                print 'updating percent to ' + str(downloadedPercent)
                if os.path.getsize(mypath) > (minSize * 1000000):
                    print 'file is now > ' + str(minSize)  
                    dProgress.close()
                    if dlThread.isAlive():
                        listitem=Item_Meta(name)
                              
                        play_with_watched(mypath, listitem, '')
                
                        #xbmc.Player().play(mypath, listitem)
                        addDownloadControls(name,mypath, listitem)
                        break
                print 'downloaded file isnt big enough yet'
                xbmc.sleep(1000)

To try and fix the issue where you run out of downloaded video to stream (and it kills the stream) could we monitor the percentWatched and make sure it is over the percentDownloaded then wait for a while? I know that isn't an exact match, but it might at least keep it close...

Good idea!

Now I'm thinking maybe we could even push that idea further and take it along the same way xbmc works with streams - gets file size, download speed and calcs how much buffer is needed

Though lets try make sure we don't spend too much on this one, it would only happen on those odd times when you get a megaupload source that doesn't want to send fast enough, majority of time it will go no problems..

Edit - filesize *might* be already be passed back in the megaupload routines when it does the login, if not that might be the best place to add it


- Eldorado - 2011-12-04 18:46

rvas18 Wrote:Eldorado, have you considered adding your git to your sig? It drives me nuts digging through the thread everytime i want to review changes. I guess im to lazy to bookmark

Don't want to make it public everywhere yet Smile

I'll start posting a link to it whenever I push updates


- zpanderson - 2011-12-05 19:17

So what have you decided on the whole download_and_watch wait functionality. I can help more on it if you let me know what needs to be done.


- Eldorado - 2011-12-05 23:42

zpanderson Wrote:So what have you decided on the whole download_and_watch wait functionality. I can help more on it if you let me know what needs to be done.

Lets put that one on the back burner for now, maybe see how it goes after release and put some time into it if we get a lot of users reporting problems?

For now I'm hoping we start doing some solid testing of all the new features.. in the background it would be nice to try and integrate your stacked sources with download&watch - if not we will need D&W and Video Seeking disabled if Stacked is turned on

I've done some small updates, mainly uploaded all of the fanart zips and touched up the code around them, so give those a go, watch out for your megaupload download limit as I hit mine a few times in testing and only received 1KB files (I put in a check for it)

Also, corrected the one folder that wasn't being created

Update Icefilms and Metahandlers - https://github.com/Eldorados


Side question for you guys, looking for opinions

Due to the size of these meta packs, would it be better to have even more options on what is downloaded/installed?

eg. Currently if meta is turned on you MUST download the DB (absolutely necessary no matter what) and the Movies + TV Show covers which equals around 900MB

Should I add more options where they can select to have Movie or TV covers? Possibly basic meta data is just the DB and no graphics?

So options would be:

Enable Meta Data - this would download just the DB
.....Movie Covers
.....TV Covers
.....Movie Fanart
.....TV Fanart


- zpanderson - 2011-12-06 05:25

That all sounds good... I would think the small fix I put in for the download & watch would be good but that's just on me (I like % better than a set time).

I'll look into stacked sources + dl&w and see what I can come up with...

I think for now on the Meta options we should give it a try and see if people complain or not.

Finally, could you go ahead and put this into the repo as an alpha or something to make it easier Wink

[EDIT] So far it looks like I'm not able to install the metacontainers. Didn't really look into it much yet but here's my log.

Code:
21:58:07 T:1764  NOTICE: Loading sqlite3 as DB engine
21:58:07 T:1764  NOTICE: Loading sqlite3 as DB engine
21:58:07 T:1764  NOTICE: ---------------------------------------------------------------------------------------
21:58:07 T:1764  NOTICE: Removing previous work folder: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork
21:58:08 T:1764  NOTICE: ==========================PARAMS:
                                            URL: None
                                            NAME: None
                                            MODE: None
                                            IMDBNUM: None
                                            MYHANDLE: 1
                                            PARAMS: []
21:58:08 T:1764  NOTICE: Found and deleted 0 empty dirs.
21:58:08 T:1764  NOTICE: Found and deleted 0 empty dirs.
21:58:08 T:1764  NOTICE: Account: login succeeded
21:58:08 T:1764  NOTICE: Table movie_meta initialized
21:58:08 T:1764  NOTICE: Table tvshow_meta initialized
21:58:08 T:1764  NOTICE: Table season_meta initialized
21:58:08 T:1764  NOTICE: Table episode_meta initialized
21:58:08 T:1764  NOTICE: Table addons initialized
21:58:08 T:1764  NOTICE: ---------------------------------------------------------------------------------------
21:58:08 T:1764  NOTICE: Removing previous work folder: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork
21:58:08 T:1764  NOTICE: Looking up in local cache for addon id: plugin.video.icefilms
21:58:08 T:1764  NOTICE: SQL Select: SELECT * FROM addons WHERE addon_id = 'plugin.video.icefilms'
21:58:08 T:1764  NOTICE: No match in local DB for addon_id: plugin.video.icefilms
21:58:17 T:1764  NOTICE: Download URL: http://www.megaupload.com/?d=YD7U5M71
21:58:18 T:1764  NOTICE: FILEPATH:
21:58:18 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.zip
21:58:18 T:1764  NOTICE: Downloading zip: http://www438.megaupload.com/files/316951d49a444e98372387f5ebd56881/video_cache.zip
21:58:18 T:1764  NOTICE: waiting 26 secs
21:58:18 T:1764   ERROR: C:\Users\HT\AppData\Roaming\XBMC\addons\plugin.video.icefilms\default.py:1966: DeprecationWarning: integer argument expected, got float
                                              pDialog.update(percent,' '+text,remaining_display)
21:58:44 T:1764  NOTICE: done waiting
21:58:48 T:1764  NOTICE: !!!!handling meta install!!!!
21:58:48 T:1764  NOTICE: Attempting to install type: database  path: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.zip
21:58:48 T:1764  NOTICE: Extracting C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.zip to C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork
21:58:48 T:1764  NOTICE: ************* Error: File size is too small
21:58:53 T:1764  NOTICE: Inserting records into table: movie_meta
21:58:53 T:1764  NOTICE: SQL Insert: INSERT OR IGNORE INTO movie_meta SELECT * FROM work_db.movie_meta
21:58:53 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.db
21:58:53 T:1764  NOTICE: ************* Error attempting to insert into table: movie_meta with error: no such table: work_db.movie_meta
21:58:53 T:1764  NOTICE: Inserting records into table: tvshow_meta
21:58:53 T:1764  NOTICE: SQL Insert: INSERT OR IGNORE INTO tvshow_meta SELECT * FROM work_db.tvshow_meta
21:58:53 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.db
21:58:53 T:1764  NOTICE: ************* Error attempting to insert into table: tvshow_meta with error: no such table: work_db.tvshow_meta
21:58:53 T:1764  NOTICE: Inserting records into table: season_meta
21:58:53 T:1764  NOTICE: SQL Insert: INSERT OR IGNORE INTO season_meta SELECT * FROM work_db.season_meta
21:58:53 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.db
21:58:53 T:1764  NOTICE: ************* Error attempting to insert into table: season_meta with error: no such table: work_db.season_meta
21:58:53 T:1764  NOTICE: Inserting records into table: episode_meta
21:58:53 T:1764  NOTICE: SQL Insert: INSERT OR IGNORE INTO episode_meta SELECT * FROM work_db.episode_meta
21:58:53 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\video_cache.db
21:58:53 T:1764  NOTICE: ************* Error attempting to insert into table: episode_meta with error: no such table: work_db.episode_meta
21:58:53 T:1764  NOTICE: Download URL: http://www.megaupload.com/?d=X8CMRAK2
21:58:53 T:1764  NOTICE: FILEPATH:
21:58:53 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\movie_covers.zip
21:58:53 T:1764  NOTICE: Downloading zip: http://www719.megaupload.com/files/6cabceb1c786d1fd0c2863af37356aad/movie_covers.zip
21:58:53 T:1764  NOTICE: waiting 26 secs
21:59:19 T:1764  NOTICE: done waiting
22:00:38 T:1764  NOTICE: !!!!handling meta install!!!!
22:00:38 T:1764  NOTICE: Attempting to install type: movie_images  path: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\movie_covers.zip
22:00:38 T:1764  NOTICE: Extracting C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\movie_covers.zip to C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\m​eta_cache\movie
22:00:38 T:1764  NOTICE: ************* Error: File size is too small
22:00:38 T:1764  NOTICE: Download URL: http://www.megaupload.com/?d=QJMWHCQW
22:00:39 T:1764  NOTICE: FILEPATH:
22:00:39 T:1764  NOTICE: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\tv_covers.zip
22:00:39 T:1764  NOTICE: Downloading zip: http://www47.megaupload.com/files/7f961c4fd4bb03032eeaddcc194d6a20/tv_covers.zip
22:00:39 T:1764  NOTICE: waiting 26 secs
22:01:05 T:1764  NOTICE: done waiting
22:03:46 T:1764  NOTICE: !!!!handling meta install!!!!
22:03:46 T:1764  NOTICE: Attempting to install type: tv_images  path: C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\tv_covers.zip
22:03:46 T:1764  NOTICE: Extracting C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\w​ork\tv_covers.zip to C:\Users\HT\AppData\Roaming\XBMC\userdata\addon_data\script.module.metahandler\m​eta_cache\tvshow
22:03:46 T:1764  NOTICE: ************* Error: File size is too small
22:03:46 T:1764   ERROR: Error Type: <type 'exceptions.NameError'>
22:03:46 T:1764   ERROR: Error Contents: global name 'get_cover_zip' is not defined
22:03:46 T:1764   ERROR: Traceback (most recent call last):
                                              File "C:\Users\HT\AppData\Roaming\XBMC\addons\plugin.video.icefilms\default.py", line 3329, in <module>
                                                CATEGORIES()
                                              File "C:\Users\HT\AppData\Roaming\XBMC\addons\plugin.video.icefilms\default.py", line 452, in CATEGORIES
                                                Startup_Routines()
                                              File "C:\Users\HT\AppData\Roaming\XBMC\addons\plugin.video.icefilms\default.py", line 420, in Startup_Routines
                                                if meta_setting=='true': ContainerStartup()
                                              File "C:\Users\HT\AppData\Roaming\XBMC\addons\plugin.video.icefilms\default.py", line 334, in ContainerStartup
                                                elif get_db_zip==False or get_cover_zip==False:
                                            NameError: global name 'get_cover_zip' is not defined
22:03:46 T:5116   ERROR: XFILE::CDirectory::GetDirectory - Error getting plugin://plugin.video.icefilms
22:03:46 T:5116   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.icefilms) failed

Line 334 needs to be updated to "elif get_db_zip==False or get_movie_cover_zip==False or get_tv_cover_zip==False:".

But that doesn't explain why my installation is failing...

All of my files look to be the correct size...

movie_covers.zip - 265,407
tv_covers.zip - 529,585
video_cach.zip - 6,507

but they all say too small...

added a print to see what size file it thinks it's getting...

Code:
22:29:05 T:4056  NOTICE: this is the size of the file... 6662180
22:29:05 T:4056  NOTICE: ************* Error: File size is too small

so no clue...

LMAO... so I kept looking at this line of code and couldn't figure out what was wrong with it...

Code:
if os.path.getsize(src) < float(10000):

Change that < to > and it'll work just fine Wink


- spoyser - 2011-12-06 12:08

Search for John Bishop

Then select top item, John Bishops Britain - Episode List

Gives script error:

Code:
09:41:36 T:60560 M:1343979520  NOTICE: ==========================PARAMS:
                                            URL: http://www.icefilms.info/tv/series/3/2232
                                            NAME: John Bishop's Britain - Episode List
                                            MODE: 12
                                            IMDBNUM: False
                                            MYHANDLE: 0
                                            PARAMS: {'url': 'http%3A%2F%2Fwww.icefilms.info%2Ftv%2Fseries%2F3%2F2232', 'videoType': 'tvshow', 'imdbnum': 'False', 'mode': '12', 'name': 'John+Bishop%26%2339%3Bs+Britain+-+Episode+List+'}
09:41:36 T:60560 M:1343979520  NOTICE: http://www.icefilms.info/tv/series/3/2232
09:41:36 T:60560 M:1343979520  NOTICE: GetUrl: http://www.icefilms.info/tv/series/3/2232
09:41:36 T:60560 M:1343979520  NOTICE: params: None
09:41:36 T:60560 M:1343979520  NOTICE: referrer: 'http://www.icefilms.info'
09:41:36 T:60560 M:1343979520  NOTICE: cookie: None
09:41:36 T:60560 M:1343979520  NOTICE: save_cookie: False
09:41:36 T:60560 M:1345458176  NOTICE: Table movie_meta initialized
09:41:36 T:60560 M:1345458176  NOTICE: Table tvshow_meta initialized
09:41:36 T:60560 M:1345458176  NOTICE: Table season_meta initialized
09:41:36 T:60560 M:1345458176  NOTICE: Table episode_meta initialized
09:41:36 T:60560 M:1345458176  NOTICE: Table addons initialized
09:41:36 T:60560 M:1345458176  NOTICE: Looking up in local cache for addon id: plugin.video.icefilms
09:41:36 T:60560 M:1345458176  NOTICE: SQL Select: SELECT * FROM addons WHERE addon_id = 'plugin.video.icefilms'
09:41:36 T:60560 M:1345454080  NOTICE: Found addon id in cache table:
09:41:36 T:60560 M:1345454080  NOTICE: {'addon_id': u'plugin.video.icefilms', 'tv_backdrops': u'false', 'covers': u'true', 'movie_backdrops': u'false'}
09:41:36 T:60560 M:1345454080  NOTICE: Retrieving TVDB ID
09:41:36 T:60560 M:1345454080  NOTICE: SQL SELECT: SELECT tvdb_id FROM tvshow_meta WHERE title = 'johnbishopx27sbritain'
09:41:36 T:60560 M:1339875328  NOTICE: Looking up season data in cache db, imdb id: None tvdb_id: None season: 1
09:41:36 T:60560 M:1339875328   ERROR: Error Type: exceptions.UnboundLocalError
09:41:36 T:60560 M:1339875328   ERROR: Error Contents: local variable 'sql_select' referenced before assignment
09:41:36 T:60560 M:1339772928   ERROR: Traceback (most recent call last):
                                              File "C:\Documents and Settings\SeanPoyser\Application Data\XBMC\addons\plugin.video.icefilms\default.py", line 3465, in ?
                                                TVSEASONS(url,imdbnum)
                                              File "C:\Documents and Settings\SeanPoyser\Application Data\XBMC\addons\plugin.video.icefilms\default.py", line 1207, in TVSEASONS
                                                season_meta = metaget.get_seasons(showname, imdb_id, season_nums)
                                              File "C:\Documents and Settings\SeanPoyser\Application Data\XBMC\addons\script.module.metahandler\lib\metahandler\metahandlers.py", line 1734, in get_seasons
                                                meta = self._cache_lookup_season(imdb_id, tvdb_id, season)
                                              File "C:\Documents and Settings\SeanPoyser\Application Data\XBMC\addons\script.module.metahandler\lib\metahandler\metahandlers.py", line 1816, in _cache_lookup_season
                                                print 'SQL Select: %s' % sql_select
                                            UnboundLocalError: local variable 'sql_select' referenced before assignment
09:41:36 T:56804 M:1342103552   ERROR: XFILE::CDirectory::GetDirectory - Error getting plugin://plugin.video.icefilms/?url=http%3A%2F%2Fwww.icefilms.info%2Ftv%2Fseries%2F3%2F2232&mode=12&name=John+Bishop%26%2339%3Bs+Britain+-+Episode+List+&imdbnum=False&videoType=tvshow
09:41:36 T:56804 M:1342103552   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.icefilms/?url=http%3A%2F%2Fwww.icefilms.info%2Ftv%2Fseries%2F3%2F2232&mode=12&name=John+Bishop%26%2339%3Bs+Britain+-+Episode+List+&imdbnum=False&videoType=tvshow) failed

Looks like it could be to do with the tvdb_id being set to None, could it be to do with the x27 code for '

Browsing normally to same show works okay, searching for other shows (eg spooks and cold feet) works okay.

D&W still doesn't work for me :-(

Code:
09:47:44 T:61312 M:1324974080  NOTICE: Creating InputStream
09:47:44 T:61312 M:1324937216   ERROR: CDVDPlayer::OpenInputStream - error opening [C:\Ice\Icefilms Downloaded Videos\TV Shows\Spooks (2002)\Spooks 10x06 (2011).avi]
09:47:44 T:61312 M:1324937216  NOTICE: CDVDPlayer::OnExit()

The file is there and VLC is able to play it.

I don't know why but on my setup XBMC (Windows XP) is unable to play any file whilst it is currently being downloaded even if I just browse to as a source and try to play it. Is there a Windows setting or environmental variable to allow it?


- Eldorado - 2011-12-06 17:56

Ahhhh.. stupid errors! Smile

zpanderson - fixed both!

spoyser - this one sucks, it looks like the x27 is causing the issue.. if it was properly replaced then the lookup would have worked... need to track down in the icefilms code why the x27 isn't being removed

edit - got it fixed, but has exposed a small hole in metahandlers and the lack of an imdb num... erg

Also, can you give ndeshmukh's mod a try and see if you can use either File Based or Env Based D&W? I'm hoping we don't need to implement both..


I have one more change I need to make to metahandlers, specifically the addons table... going to add 2 columns to account for Movie Covers and TV Covers instead of just one field 'Covers' as it is now, as well as a LastUpdated field so addons can track when the last meta pack release was done and easily send updates. Nothing more will be done within Icefilms to use these, but just want the foundation to be laid and set within metahandlers

How would you guys like alpha testing to be done? Should I set it up into a repo so that you get the updates automatically?


Also.. just thinking of this after reading the main page post - http://xbmc.org/natethomas/2011/12/03/addon-authors-please-prepare-for-eden/

For the most part all code is up to par with Eden, except for the python api version, if you look in addon.xml it is still set to 1.0

Should we restrict 1.1.0 to only Pre-Eden users? I think some of the small issues I've had with metahandlers is due to using the 1.0 api.. and sounds like Eden isn't too far away


- spoyser - 2011-12-06 18:54

Eldorado Wrote:spoyser - this one sucks, it looks like the x27 is causing the issue.. if it was properly replaced then the lookup would have worked... need to track down in the icefilms code why the x27 isn't being removed

Since it only happens from the search results I guess the x27 code isn't being fixed up in the find_meta_for_search_results function (could it be missing a call to CLEANUP?)

Eldorado Wrote:Also, can you give ndeshmukh's mod a try and see if you can use either File Based or Env Based D&W? I'm hoping we don't need to implement both..

I'll give it a try, can you let me know where it can be downloaded from?
Edit: Found it

Eldorado Wrote:How would you guys like alpha testing to be done? Should I set it up into a repo so that you get the updates automatically?

Automatic updates sounds good to me, I have that feature switched off by default (stops my personal hacks getting overwritten!) but selecting update in XBMC is still easier than downloading them manually from git.


- Eldorado - 2011-12-06 20:25

spoyser Wrote:Since it only happens from the search results I guess the x27 code isn't being fixed up in the find_meta_for_search_results function (could it be missing a call to CLEANUP?)

Automatic updates sounds good to me, I have that feature switched off by default (stops my personal hacks getting overwritten!) but selecting update in XBMC is still easier than downloading them manually from git.

Found the fix needed to be where it scrapes for seasons, and then again for episodes.. the name wasn't being cleaned before passing into metahandler, working good now


I also switched to use python api 2.0 and it fixed some of my issues I had to work around, specifically datetime formatting.. now just need to figure out how to hide the 1.1.0 upgrade from non-Eden users as I thinking this is the way to go