EPG (TV-Guide) data-store code development discussion for the unified PVR front-end

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
jmarshall Offline
Team-XBMC Developer
Posts: 26,181
Joined: Oct 2003
Reputation: 175
Post: #16
Agreed with your analysis. A simple, efficient API to expose the db is the way to go, and the GUI just pulls from the db.

A simple RefreshGUI command in the API can be used to push updates to the GUI as needed (or this can be automated by the API itself).

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
alcoheca Offline
Team-XBMC Developer
Posts: 265
Joined: Feb 2008
Reputation: 6
Post: #17
I'd still like to hear other views on this, however for my proposal I'll probably say this is how it will work. (I don't have a huge amount of time over the weekend and proposal needs to be in by the 31st.)

RE Gamester17's question about where to store all the "other" data....I can't answer that question, but perhaps SQL can let us link entries in little tables (Rec. status, schedule status, etc) with entries in the big tables (EPG).

Not sure it would make sense to add empty fields to the EPG table with hundreds of channels, or does SQL compress that sort of thing well?

Is accessing one table with a transaction faster than two tables?

If we have 8 channels on screen with 2-3 hours of programming that could be ~48 individual programme, "Rec. status", "Schedule Status" queries while the user is waiting for screen to update.

Getting this as snappy as possible on the Xbox is a must.
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,181
Joined: Oct 2003
Reputation: 175
Post: #18
SQLite it quick with multiple tables, assuming the query hits them at the same time and there is good unique links being used. The key to a good DB design (and I'm by no means a DB designer!) is specifying the data you have and exactly how you need to be able to access it, and then arranging the tables so that the queries will be quick. A lot of prototyping can be done in programs such as sqlitespy and so on.

Even for 2000 channels over a few hours isn't a huge amount of data that needs to be displayed onscreen at a time, and we can always background-fetch the data as needed if necessary. The more detailed information (plot/cast etc.) can then be retrieved on demand (we'd have a unique show id at that point, so the lookup would be quick).

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
alcoheca Offline
Team-XBMC Developer
Posts: 265
Joined: Feb 2008
Reputation: 6
Post: #19
jmarshall Wrote:The key to a good DB design (and I'm by no means a DB designer!) is specifying the data you have and exactly how you need to be able to access it, and then arranging the tables so that the queries will be quick. A lot of prototyping can be done in programs such as sqlitespy and so on.

That's one of the next things I was going to look at, the data structure. I'll try and get my head into sql and get some bechmarking on the go, however if there's any sql gurus out there with an interest feel free to beat me to it!

edit: in fact now I think a little longer on this, I have no time right now to bench mark SQL queries on large data sets. I haven't got far with my qualifying task yet... Frown
(This post was last modified: 2008-03-28 01:48 by alcoheca.)
find quote
Gamester17 Offline
Team-XBMC Forum Moderator
Posts: 10,523
Joined: Sep 2003
Reputation: 9
Location: Sweden
Big Grin   
Post: #20
alcoheca Wrote:I haven't got far with my qualifying task yet... Frown
Concentrate on getting the proposal written and submitted now before the 31st of March, the qualification task does not have to be completed until the 10th of April (though it should preferably be submitted a few days before then so that it can be reviewed and reworked if needed, so that it can be committed to out SVN in time).

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
alcoheca Offline
Team-XBMC Developer
Posts: 265
Joined: Feb 2008
Reputation: 6
Post: #21
I've not kept this thread up to date, but anyone watching the wiki page will have noticed that my application is written and was submitted on 30/03/08

Since then I've had to concentrate on uni work - 3 projects nearing completion deadline - and getting my head around the qualificiation task.

I've downloaded sqlitespy, and am currently creating a database to test transaction speed with large data sets. I'll need to read up on db design before I can take much from these trials, anyone with more experience in this please voice your thoughts on this.
find quote
dubstar_04 Offline
Senior Member
Posts: 228
Joined: May 2008
Reputation: 0
Location: Warrington, UK
Post: #22
Is there any news on development of this?
find quote
hotstyle765 Offline
Junior Member
Posts: 5
Joined: May 2008
Reputation: 0
Post: #23
dubstar_04 Wrote:Is there any news on development of this?

I second this... Is there any update on development?
find quote
dubstar_04 Offline
Senior Member
Posts: 228
Joined: May 2008
Reputation: 0
Location: Warrington, UK
Post: #24
Is this project still underdevelopment?
find quote
alcoheca Offline
Team-XBMC Developer
Posts: 265
Joined: Feb 2008
Reputation: 6
Post: #25
Yes, I should leave more status updates here, otherwise the moss sets in, people start worrying Smile

sqlite stuff: there's code in my subversion branch that is the basis for CTVDatabase - the class that handles all access to the sqlite file we need to store all the schedule data. As it is I can fill it and retrieve the listings back channel by channel. Of course we need more access methods, such as searching functions, filtering by field etc. Speed wise, it's a painful process:
Quote:m_database.GetDataEnd() took 828 ms to return
m_database.GetChannels() took 15 ms to return
CGUIWindowEPG::GetGridData returned 860 items in 1836 ms (2 days of 35 channels)
CGUIEPGGridContainer::UpdateItems completed successfully in 172 ms
GetDataEnd is a stupid method I use to find out what date the data runs out in the db. It will need to be replaced by a static value stored somewhere, so that can be ignored; nearly 2 seconds to get 860 items is pretty shocking however. Improvement needed here as this is performed each time you flick to the TV guide window. I know it's possible to cache this data and/or improve the query and design of the db.

GUI TV guide: There's code in svn for a window (CGUIWindowEPG) that retrieves 2 days of data as a 2d vector of FileItems, builds a grid of GUIListItems and which can be navigated reasonably well. This has been a really tough thing to get working, tough because I've made mistakes, have been improving and reshaping a lot of code, and primarily because I came to this with a very limited understanding of XBMC's GUIlib. (I wouldn't recommend my approach for time limited projects..) There are some nice things in it, such as semi-intelligent navigation, proper horizontal scrolling rather than page flipping; the basis of a reasonably sophisticated container class. It's needless to say, there's a lot more to do, and challenges are ahead - but I'm still enjoying myself Wink
find quote
rwparris2 Offline
Team-XBMC Python Developer
Posts: 1,333
Joined: Jan 2008
Reputation: 2
Location: US
Post: #26
Just wanted to pop in and say thank you and good luck with the project... we're all very excited to see development on this front (at least I am :nodSmile
find quote
dubstar_04 Offline
Senior Member
Posts: 228
Joined: May 2008
Reputation: 0
Location: Warrington, UK
Post: #27
rwparris2 Wrote:Just wanted to pop in and say thank you and good luck with the project... we're all very excited to see development on this front (at least I am :nodSmile

I second this. I check the forums every day to see if there is any progress with your project.

I would also like to say thank you for all you effort and i sincerely hope that you gain everything from the project / experience that you wish for.

Would you be able to post any screenshots of progress and you go?

Many thanks,

Dubstar_04
find quote
alcoheca Offline
Team-XBMC Developer
Posts: 265
Joined: Feb 2008
Reputation: 6
Post: #28
in case you miss it there's a screenshot and some info on the skinning aspects in the Skinning Forum - http://forum.xbmc.org/showthread.php?p=2...post205811
find quote
dubstar_04 Offline
Senior Member
Posts: 228
Joined: May 2008
Reputation: 0
Location: Warrington, UK
Post: #29
WOW!!

Its so great to see that you have made so much progress!!

Keep it coming!!
find quote
Jeroen Offline
Skilled Skinner
Posts: 2,463
Joined: Feb 2008
Reputation: 19
Location: The Netherlands
Post: #30
I think this will become a very important project for XBMC. We all know XBMC is simply the best mediacenter around, but in this particular area it was clearly behind others. With this it will surely catch up in just about the last area it had any catching up to do Tongue
Keep it up Smile
find quote