EPG (TV-Guide) data-store code development discussion for the unified PVR front-end

Hopefully this will serve a place to continue this discussion in the User Area concerning the methods for storing, accessing and displaying the EPG data within the XBMC application. (Note that I'm not talking about the specific mechanisms that the individual plugins will use to communicate with the PVR server. There will need to be threads set up for those separately, when the time comes).

This is partly to encourage the open discussion of these issues in the development of a XBMC-class EPG, but also because I need this sort of information for my proposal!

I'll start the ball rolling with thoughts on the EPG data-store.

So a major issue to iron out is how the EPG data is stored in XBMC before being displayed by the GUI, and how the data will be populated by the 'client' role of the XBMC app. This will have to be a unified solution for the multitude of different PVR applications out there.

There has been a suggestion that XML is used, for a number of reasons:

• It's likely that XML will be one of the methods for Server->Client communication of EPG data
• Some people will want to use XMLTV 'grabbers' to populate their EPG
• Python scritps could easily output an XML file
• The previous work done on the XMLTV GUI could be used as a base to work from
  

I can certainly see XML being used by the client part when that is how the server transfers the EPG, but I don't think it's suitable as the internal data store.

Perhaps using the SQL db which is already being used for Music and Video libraries would be more sensible. The 'client' role would populate the EPG related tables and the GUI would just query the db for the data.

Someone with more knowledge of the performance benefits of this might want to add their views here... It's quite possible the EPG would be updated every minute, particularly if the user was taking advantage of EIT for changes in program start times etc. With the possibility for very large EPGs coming off of digital satellite, I imagine SQL querying to be far more efficient than XML parsing (on the fly).

Plus the Video library code could be easily modified to include the TV recordings as a root folder.

what does anyone think?

I guess it depends on:

1. Amount of data - can we get some info on typical sizes?
2. Frequency of data updates (i.e. how often do we need to add to the datastore?).
3. Frequency of reading the EPG from within XBMC (i.e. how often do we need to check the datastore for refresh, or should this be pushed from 2?)

With that info, it should be easy to setup some test cases to see whether XML parsing will suffice in terms of speed and memory usage and so on, or whether we need to sql-ize it.

Cheers,
Jonathan

I'll try and come up with some values for typical data sizes before I go to sleep tonight. If we imagine an average 14 day data set with ~200 channels it's probably not a massive amount but I'm thinking more along the lines of polling the data within xbmc which will require a more sophisticated storage system.

Plus it seems conducive with getting recordings into Videos section..

In respect to the frequency of updates, if the EPG is to be truly 'live' then I imagine frequent transactions from the client role - even as much as once a minute?

1.6mb here from shepherd epg australia.

hi nick,

could you let us know the number of channels & days ?

oh thats only 10 or so channels as well. its large because it has all the descriptions, actors etc

sorry I posted before your reply. 7 days. Ill send you the xml if you want. tried to put it on pastebin but it was too big.

well I'm starting to get tired now but I have some results

UK RadioTimes: 220 channels and 14 days grab = 46MB XML file with the following schema:

[HTML] <programme start="20080408090000 +0100" stop="20080408093000 +0100" channel="digital.scotland.bbc2.bbc.co.uk">
<title>The Story of Tracy Beaker</title>
<desc lang="en">Drama series about a young girl who is looking for a foster home.</desc>
<credits>
<actor>Dani Harmer</actor>
<actor>Montanna Thompson</actor>
<actor>Chelsie Padley</actor>
<actor>Joe Starrs</actor>
<actor>Sonny Muslim</actor>
<actor>Jay Haher</actor>
</credits>
<category lang="en">Children</category>
<video>
<aspect>16:9</aspect>
</video>
<subtitles type="teletext" />
</programme>
[/HTML]


Having that many channels seems like madness to me (who watches very little television) but the 14 days must be a realistic average, as people will want to plan their recordings for holidays away etc.

Right. Parsing a 46MB file is completely unrealistic on xbox, so an alternative storage system will certainly be required.

Cheers,
Jonathan