[RELEASE] TV Guide - with XMLTV and streaming support

timpinkawa · 2012-03-08 04:07

twinther Wrote:btw timpinkawa, please remove the buggalo.SUBMIT_URL or change it to 'http://buggalo.xbmc.info/submit.php' - I'm getting a few bug reports from your code and it's difficult enough to keep track of them as it is :-)

Done.

Current users should update and spare Tommy's bug report server any more traffic. Laugh

I have a 150MB (7.3MB compressed) XMLTV file I've been testing with. If you're interested I can get you a copy one way or another.

twinther · 2012-03-08 12:11

timpinkawa Wrote:I have a 150MB (7.3MB compressed) XMLTV file I've been testing with. If you're interested I can get you a copy one way or another.

Yeah, that would be great. can you upload to dropbox or some other place?
You can PM me the link if it's private, otherwise I think my mailbox should accept 7 mb

Br.
Tommy

spoyser · 2012-03-09 13:26

@twinther

Great plugin! I've been using it in the UK with data from the RadioTimes web site (via a third party app to create the XMLTV file and it works great)

I've also been looking into optimising the speed (on top of the changes from timpinkawa) it and have a few comments.

In the getProgramList() you can make use of a member variable dictionary to cache the program lists to memory giving virtually instant (< 0.1sec) retrieval of the program list (although obviously this could potentially use a lot of memory, in the UK 14 days of 50 channels is about 18000 programs which increases memory usage of XBMC by about 12MB, so not a big deal for this size of data)

(I've just noticed you make use of the start and end dates in the latest code when querying the database, I haven't found that necessary when caching the data to memory)

You've updated the code quite a lot but something like this should do the trick:

Code:

#in Source __init__

 self.cache = dict() 

#in _retrieveProgramListFromDatabase(self, channel, startTime):

key = '%s-%s' % (self.KEY, channel.id.replace('/', ''))

if key in self.cache:

    return self.cache[key]

programList = list()        

c = self.conn.cursor()        

c.execute('SELECT * FROM programs WHERE channel=? AND source=?', [channel.id, self.KEY])        

for row in c:

    program = Program(channel, row['title'], row['start_date'], row['end_date'], row['description'], row['image_large'], row['image_small'])

    programList.append(program)

self.cache[key] = programList

return programList

#the appropriate key in self.cache will also need to be cleared whenever the database is updated

Also just a couple of minor bugs, in gui.py when you set the position of the current time you don't take into account the day so you get the line at the "NOW" time regardless of the day;

Code:

# move timebar to current time

timeDelta = datetime.datetime.today() - self.viewStartDate

c = self.getControl(4100)

(x, y) = c.getPosition()

if timeDelta.days == 0:

    c.setPosition(self._secondsToXposition(timeDelta.seconds), y)

else:

    c.setPosition(-100, y)

In _scheduleNotification in notification.py again you ignore the day so the notification can appear 24 hours early;

Code:

t = self._timeToNotification(program)

 timeToNotification = ((t.days * 86400) + t.seconds) / 60

 if timeToNotification < 0:

     return

Also in the Notification class in the scheduleNotifications function you can grab the channel list just once and then
pass it into _processSingleNotification function (even better is to have the list as a member variable)

And in the onAction() of the TVGuide class, if you add this

Code:

elif action.getId() == 159: #KEY_HOME

    self.viewStartDate = datetime.datetime.today()

    self.viewStartDate -= datetime.timedelta(minutes = self.viewStartDate.minute % 30)

    self.onRedrawEPG(self.page, self.viewStartDate)

Then pressing the Home key will return the guide to the current time, which I use a lot!

HTH

shaktoo · 2012-03-09 14:32

@ Spoyser, Could you help out a little bit if possible. I have been trying to make the XMLTV file so I can get this working but so far have been unable to . I think you are talking about using mc2xml ... Could you do a step by step tut if possible or just PM me

spoyser · 2012-03-09 14:54

shaktoo Wrote:@ Spoyser, Could you help out a little bit if possible. I have been trying to make the XMLTV file so I can get this working but so far have been unable to . I think you are talking about using mc2xml ... Could you do a step by step tut if possible or just PM me

Actually I've been using XMLTV GUI Grabber (see here http://www.birtles.org.uk/xmltv/). Is is fairly straight forward to use but if you get stuck let me know via PM

(And for the freeview channel logos you can grab them from here: http://www.freeview.co.uk/Channels )

timpinkawa · 2012-03-10 03:16

spoyser Wrote:In the getProgramList() you can make use of a member variable dictionary to cache the program lists to memory giving virtually instant (< 0.1sec) retrieval of the program list (although obviously this could potentially use a lot of memory, in the UK 14 days of 50 channels is about 18000 programs which increases memory usage of XBMC by about 12MB, so not a big deal for this size of data)

I would advise some caution on this. With US Digital Cable, mc2xml generates files with slightly over 200,000 programs (default settings).

spoyser · 2012-03-10 11:18

timpinkawa Wrote:I would advise some caution on this. With US Digital Cable, mc2xml generates files with slightly over 200,000 programs (default settings).

Absolutely agree with you on that, although based on my (very) rough estimate with every channel cached that would still only be about 150MB of memory.

In addition the way I have implemented it, a channel only gets cached if _retrieveProgramListFromDatabase is called for it so not all channels will necessarily get cached anyhow.

You could also limit the dictionary to say 100 channels, removing the 1 when the 101 item gets added.

I would suggest at least giving it a try to see how it performs, like I said with the UK dataset the access rate is instant (after the initial retrieval/caching), in fact it is even faster than the EPG built into my STB!

spoyser · 2012-03-12 17:44

Looks like buggalo.py is missing from the latest download (twinther-script.tvguide-v1.2.2-13-g32d41b3) :-(

twinther · 2012-03-12 19:23

(2012-03-12 17:44)spoyser Wrote: Looks like buggalo.py is missing from the latest download (twinther-script.tvguide-v1.2.2-13-g32d41b3) :-(

Yeah, it's a separate module now, you can get it here:
http://mirrors.xbmc.org/addons/eden/scri...e.buggalo/

Br.
Tommy

twinther · 2012-03-16 11:48

I have uploaded a test release with the changes I have been working on. The new version is 1.2.90 and will become 1.3.0 once it is stable enough. The easiest way to get it is to install my beta repository from here: http://tommy.winther.nu/xbmc/
You can also download the zip from there as well, if you do that remember to install the Buggalo Exception Collector as well.

The major changes in this version are:
- Added French translation
- Use script.module.buggalo for error reporting
- Cache EPG data to local SQLite database (based on work by timpinkawa)
- Added option to edit channel order and visibility
- Automatically detects changes in settings and reloads EPG data as needed
- Improved XMLTV parsing (thanks timpinkawa)
- Added initial channel surfing support
- Press Info for OSD, navigate with Up, Down, Left, Right, Channel + and -
- Press ContextMenu to minimize video and show EPG
- Added support for mouse wheel scrolling (will probably crash XBMC - see trac:12797)

I think it's time for some screenshots again...
[Image: script.tvguide-1.3.0a.png]

twinther Senior Member Posts: 294 Joined: Sep 2004 Reputation: 7 Location: Denmark	2012-03-08 12:11 Post: #102 timpinkawa Wrote:I have a 150MB (7.3MB compressed) XMLTV file I've been testing with. If you're interested I can get you a copy one way or another. Yeah, that would be great. can you upload to dropbox or some other place? You can PM me the link if it's private, otherwise I think my mailbox should accept 7 mb Br. Tommy
	find quote

shaktoo Fan Posts: 642 Joined: Mar 2010 Reputation: 3 Location: in LIMBO	2012-03-09 14:32 Post: #104 @ Spoyser, Could you help out a little bit if possible. I have been trying to make the XMLTV file so I can get this working but so far have been unable to . I think you are talking about using mc2xml ... Could you do a step by step tut if possible or just PM me
	find quote

spoyser Senior Member Posts: 169 Joined: Oct 2011 Reputation: 10	2012-03-12 17:44 Post: #108 Looks like buggalo.py is missing from the latest download (twinther-script.tvguide-v1.2.2-13-g32d41b3) :-(
	find quote

twinther Senior Member Posts: 294 Joined: Sep 2004 Reputation: 7 Location: Denmark	2012-03-12 19:23 Post: #109 (2012-03-12 17:44)spoyser Wrote: Looks like buggalo.py is missing from the latest download (twinther-script.tvguide-v1.2.2-13-g32d41b3) :-( Yeah, it's a separate module now, you can get it here: http://mirrors.xbmc.org/addons/eden/scri...e.buggalo/ Br. Tommy
	find quote