Bug LiveTV channels and EPG not cached
#1
It looks like all Live TV data is not cached on clients. If raspbmc starts up a useless progress bar is shown that downloads the channels, epg and recordings list.this can take a serious amount time. While the progress bar is shown the PVR plugin is not loaded. I cannot start live tv while this is running.

I think this should only happen once and than incremental.
- Channels are not changing in 99% of all cases.
- Downloading EPG data for 14 days with a lage number of channels can take extreme long.
- I have 600 recordings on my WMC. No idea why it takes a long time do make a dir listing of this.

Proposed changes:
1. Cache all this data locally and only sync differences in background only.
2. Start PVR with system boot, do not wait until EPG and Record list has been downloaded. To start PVR only the channel list is required. EPG can be a background task and recordings, too. Currently I cannot start LiveTV for about 5 minutes as it need to download all stuff. This suxx a lot.
3. Allow disabling the progress bar. I do not care if EPG is outdated or empty on startup. I may like starting a movie only - no live tv, too. Windows Media Center is also empty on first start up and fills up over time. All stuff is updated without telling the users. In most I suspect that today or next 4h are most important and the rest can be done in background.
4. If network connection to backend get's broken while downloading - xbmc gets stuck. You cannot reboot raspbmc in this case. As example I waited 30 minutes and then powercycled the raspi as it stopped somewhere in between channels or epg or recordings. I was made to do this this very very often as the nano wlan stick was not the best idea here (weak connection). This situation is not propperly handled by xbmc if it comes to live tv. If network get's broken while viewing a movie I get a timeout message box. I expect the same here.

Please install Windows Media Center and review how smooth it works and than you get an idea how a good media centers must work. Don't hurt me for this, please. XBMC Live TV is not ready for production my point of view. I tried to migrate to xbmc, but this looks like a very bad idea to me.
Reply
#2
the channels are epg data is cached, your pi is just terribly slow to load it from the sd card.

(2014-01-14, 23:38)marc.bau Wrote: 1. Cache all this data locally and only sync differences in background only.
that's exactly what is happening already
(2014-01-14, 23:38)marc.bau Wrote: 2. Start PVR with system boot, do not wait until EPG and Record list has been downloaded. To start PVR only the channel list is required. EPG can be a background task and recordings, too. Currently I cannot start LiveTV for about 5 minutes as it need to download all stuff. This suxx a lot.
that must be a problem with your backend or add-on. even with a slow RPi, loading things from tvheadend is pretty quick
(2014-01-14, 23:38)marc.bau Wrote: 3. Allow disabling the progress bar. I do not care if EPG is outdated or empty on startup. I may like starting a movie only - no live tv, too. Windows Media Center is also empty on first start up and fills up over time. All stuff is updated without telling the users. In most I suspect that today or next 4h are most important and the rest can be done in background.
feel free to create a PR and i might merge it. only the initial import shows a progress bar right now, and i like it Smile the same thing happens when you update your library in XBMC, it will also show a progress bar then. this is the same

(2014-01-14, 23:38)marc.bau Wrote: 4. If network connection to backend get's broken while downloading - xbmc gets stuck. You cannot reboot raspbmc in this case. As example I waited 30 minutes and then powercycled the raspi as it stopped somewhere in between channels or epg or recordings. I was made to do this this very very often as the nano wlan stick was not the best idea here (weak connection). This situation is not propperly handled by xbmc if it comes to live tv. If network get's broken while viewing a movie I get a timeout message box. I expect the same here.
this isn't handled properly by the add-on that you're using
(2014-01-14, 23:38)marc.bau Wrote: Please install Windows Media Center and review how smooth it works and than you get an idea how a good media centers must work. Don't hurt me for this, please. XBMC Live TV is not ready for production my point of view. I tried to migrate to xbmc, but this looks like a very bad idea to me.
so you ran WMC on similar hardware as the pi? don't think so.

PVR in XBMC is fine, it's the slow hardware that you're using that's the problem. some code can definitely be improved, but the things you're mentioning are caused by your slow hardware and the amount of data (epg, recordings, channels) you're trying to load.
opdenkamp / dushmaniac

xbmc-pvr [Eden-PVR builds] [now included in mainline XBMC, so no more source link here :)]
personal website: [link]

Found a problem with PVR? Report it on Trac, under "PVR - core components". Please attach the full debug log.

If you like my work, please consider donating to me and/or Team XBMC.
Reply
#3
marc.bau I have a Pi and I am more than happy with the performance for watching live tv recordings etc you should try the openelec unofficial builds http://forum.xbmc.org/showthread.php?tid=169674.

They are very stable for me and Pi is very smooth also try a modest overclock my one is 800. Also follow these instructions http://wiki.xbmc.org/index.php?title=HOW..._USB_drive. For a very cheap device it is performing remarkable well. Plus the code is always improving. There are some improvements that pvr can improve on like record button on timeline other than that I am very happy with XBMC.
Reply
#4
(2014-01-15, 11:58)opdenkamp Wrote: the channels are epg data is cached, your pi is just terribly slow to load it from the sd card.

I have been told be several people that this is not true and I experience the same. If you reboot, all stuff is loaded from network. The maintainer of WMC plugin told me this is out of his hands. This is XBMC that does this.

(2014-01-14, 23:38)marc.bau Wrote: 1. Cache all this data locally and only sync differences in background only.
that's exactly what is happening already

I'm sorry - no. If it would - a weak/breaking network connection cannot make the complete pi stuck.

(2014-01-15, 11:58)opdenkamp Wrote:
(2014-01-14, 23:38)marc.bau Wrote: 2. Start PVR with system boot, do not wait until EPG and Record list has been downloaded. To start PVR only the channel list is required. EPG can be a background task and recordings, too. Currently I cannot start LiveTV for about 5 minutes as it need to download all stuff. This suxx a lot.
that must be a problem with your backend or add-on. even with a slow RPi, loading things from tvheadend is pretty quick

That's interresting. I will point WMC maintainer here as we tells me this is XBMC. Damn finger pointing.

(2014-01-15, 11:58)opdenkamp Wrote:
(2014-01-14, 23:38)marc.bau Wrote: 3. Allow disabling the progress bar. I do not care if EPG is outdated or empty on startup. I may like starting a movie only - no live tv, too. Windows Media Center is also empty on first start up and fills up over time. All stuff is updated without telling the users. In most I suspect that today or next 4h are most important and the rest can be done in background.
feel free to create a PR and i might merge it. only the initial import shows a progress bar right now, and i like it Smile the same thing happens when you update your library in XBMC, it will also show a progress bar then. this is the same

What is "PR"? I'm not a programmer of XBMC... I see the progress bar on every pi reboot. This means this is not an initial (one time) thing only.

(2014-01-15, 11:58)opdenkamp Wrote:
(2014-01-14, 23:38)marc.bau Wrote: 4. If network connection to backend get's broken while downloading - xbmc gets stuck. You cannot reboot raspbmc in this case. As example I waited 30 minutes and then powercycled the raspi as it stopped somewhere in between channels or epg or recordings. I was made to do this this very very often as the nano wlan stick was not the best idea here (weak connection). This situation is not propperly handled by xbmc if it comes to live tv. If network get's broken while viewing a movie I get a timeout message box. I expect the same here.
this isn't handled properly by the add-on that you're using

That's interresting. I will point WMC maintainer here as we tells me this is XBMC. Damn finger pointing.

(2014-01-15, 11:58)opdenkamp Wrote:
(2014-01-14, 23:38)marc.bau Wrote: Please install Windows Media Center and review how smooth it works and than you get an idea how a good media centers must work. Don't hurt me for this, please. XBMC Live TV is not ready for production my point of view. I tried to migrate to xbmc, but this looks like a very bad idea to me.
so you ran WMC on similar hardware as the pi? don't think so.

This was not about the PI, it is about how smooth the interface of WMC looks like and how the XBMC looks like and behave. Showing RSS may has something useful for a small number of people, but is something I disable immediately after install. That I see the progress bar on top right at boot time is also not happen on WMC, this are just background tasks. There are toooo many things in XBMC that are forefront tasks where it shouldn't be. This stuff should be disabled by default and only enabled if I'd like to see them. I guess some of these are just because developers do not trust their code and do not like to explain everyone how to enable it just to see if there is a problem or not. Smooth behavior - is all background task that has no benefit for the user who just like to see a movie or live tv. Every bobbling of any technical stuff is simply not required at all as long as I'm not a technician.

(2014-01-15, 11:58)opdenkamp Wrote: PVR in XBMC is fine, it's the slow hardware that you're using that's the problem. some code can definitely be improved, but the things you're mentioning are caused by your slow hardware and the amount of data (epg, recordings, channels) you're trying to load.

This is absolutely not true. The pi is fast enough. There are just some things that need to be cached and run in background without bothering the inexperienced user. That issues also exists also high power machines.

(2014-01-15, 14:37)denz Wrote: marc.bau I have a Pi and I am more than happy with the performance for watching live tv recordings etc you should try the openelec unofficial builds http://forum.xbmc.org/showthread.php?tid=169674.

They are very stable for me and Pi is very smooth also try a modest overclock my one is 800. Also follow these instructions http://wiki.xbmc.org/index.php?title=HOW..._USB_drive. For a very cheap device it is performing remarkable well. Plus the code is always improving. There are some improvements that pvr can improve on like record button on timeline other than that I am very happy with XBMC.

Have you seen hangs without overclocking? I can try overclocking once I have some coolers installed, but this will not solve hang conditions of the interface caused by network issues and missing caching. What is the benefit of using OpenELEC compared to Raspbmc?
Reply
#5
@opdenkamp: as a user of "tvheadend"... are you using Digital Devices Cine S2 SAT card or any other with multiple tuners? Can I view/record more than one channel at the same time that is located on the same transponder with one tuner? WMC cannot - it requires one tuner per channel, but I'm looking for a software that removes this limitation and I know that MediaPortal is capable of doing this. But moving to a Linux only solution has it's own charm on the backend server.
Reply
#6
marc.bau, I second what you wrote about hiding stuff like the EPG import and alike and handle more things invisible to the user in the background. Many of the team would like to have a "zero feature release" which means no new gimmicks added to XBMC but rather focus on internal cleanup, refactoring and UX improvements. Let's see if we can convince enough devs to do this.

A note to the EPG cache you where talking about. Please note that in case you cache the EPG you'll still see an EPG import progress bar, but this one is loading the EPG from the cached DB into memory (along with some required processing of the data) rather than from the backend into memory. Personally I disabled EPG cache because this means two import cycles (first load DB, then sync with backend and write to db) and IMO loading EPG only from backend is faster. Software like MediaPortal also does the syncing, but MediaPortal does this during startup screen and thus delaying the entire application startup. In contrary we try to bring the user as fast as possible to the home screen so that he can start using XBMC while "secondary" things are still being loaded (like LiveTV stuff). This has the benefit that in case you don't want to use LiveTV atm you have much faster access to the other sections of XBMC. Otoh if you like to instantly watch TV you first have to wait on the home screen until PVR is ready. So depending on your main usecase this can be a nice or ugly behavior.
Reply
#7
(2014-01-15, 11:58)opdenkamp Wrote: PVR in XBMC is fine, it's the slow hardware that you're using that's the problem. some code can definitely be improved, but the things you're mentioning are caused by your slow hardware and the amount of data (epg, recordings, channels) you're trying to load.

This is exactly what marc was told in the serverWMC support forum. His issues are more than likely in his installation and/or his network. He was told this by the dev of swmc and others who are using similar hardware without issues. He is also quick to say that other windows issues are bugs with swmc. No shame in asking questions, god knows I've asked my share of silly ones, but you have to try to digest what is offered as an answer without your own "finger pointing".
Reply
#8
This is not the reason for things stop responding that needs a powercycle just because there is no timeout. And it's also not the reason that pvr does not start. I only showed that weak network connections will cause issues that the coders have not expected yet. Ther are missing exception handling.

I removed wlan and use powerline now. 9mb/s is really enough network speed, but this does not solve the bugs. I know what bugs are. I deal every day with tons of them and in 9/10 cases I've found a bug that will be fixed. Here it may be difficult for me as xbmc newbie to say which xbmc component is the source, but it nevertheless exists!

Only the reason that you do not like to get told about issues does not fix them. Its also not important if this is open source or not and if someone works for free or not. I'm manage a lot of open source projects and waste my spare time with gpl coding (not xbmc). Complaining that this is all volunteer work does not make anything better either.
Reply
#9
just chiming in on the "PVR functions in XBMC isnt available until channel and EPG data has loaded" which is apparently said to be something we can "fix" in our pvr addon or ServerWMC backend.

As far as my understanding of the XBMC PVR framework goes, you cannot view LiveTV until the addon has been loaded. The addon is loaded by the XBMC PVRManager class and this makes calls to load the Channels, Timers and Recordings, before the Load() function completes. You cant use the PVR functions until Load() completes, therefore this isnt something we can change in the addon. If it takes your setup 5 minutes to load these data items, then that is how long you have to wait before being able to use the PVR interface. I tend to agree that you should be able to view LiveTV as soon as you have the channels but I dont believe this is anything we can change in our PVR addon which is why it was suggested for you to put this forward as an XBMC feature request (and not a bug, incidentally). I think you have misunderstood opdenkamp who did not say that it is a problem with our addon that you cant view liveTV until all the channel/recordings/timers have been loaded, he was saying it must be a problem with the addon/backend if it is taking 5 minutes to load the data. If XBMC people want to take an enhancement request to allow more data to load in background and allow PVR functions when only the channel list is loaded, that's all well and good, but unless someone can inform me how we can do it, it's outside the scope of the pvr addon framework we work in to alter this. All he is saying is it isnt good that your setup takes 5 minutes to perform these loads.

You have what I would consider to be an extreme edge case setup with 1000 channels and 100s of recordings, yet from other posts Ive seen it seems you are running this on one of the most lightweight setups we have encountered (an Intel Atom based PC for the server side, and a raspberry pi for the client side, with an apparently flaky wireless network connection to boot although I see you have now put in powerline adapters which should be better than wireless hopefully).

To enable us to help you, please follow our guidance of producing/uploading debug logs for XBMC and ServerWMC so we might see what/where the time is being taken, and also we suggested temporarily reducing your channels/recordings to more normal values to see what difference it may make. Also I would suggest adjusting your attitude from one of entitlement to one of collaboration, we are all doing this in our spare time and owe you nothing. We want things to work well for as many people as possible and are generally helpful and constructive when we receive the same in return, or when the initial approach is "hey im having this problem could you please help me figure out what it may be" rather than "you have these bugs, this is not acceptable, etc"
pvr.wmc TV addon and ServerWMC Backend Development Team
http://bit.ly/ServerWMC
Reply
#10
I run both of my XBMC installs on Raspberry Pi hardware (Openelec Gotham) and use PVR via Mediaportal backend, ~200 channels and a few recordings takes probably a minute or two to load at XBMC startup (wired network).

One thing I have wondered, which does probably lend itself to discussion in this thread is, can the PVR framework request EPG be loaded for a specific time period?
At the moment at startup the entire EPG seems to be loaded from cache (i'm assuming it's ALL available EPG since it takes a while).

Would a possible solution for lower power boxes that are now being used for XBMC (Pi, OUYA, Cubox etc) be to load EPG from cache for "Now > 4 Hours" (or some other small value) ?

I'm assuming this would greatly speed up the PVR startup time allowing you to go straight into a fully populated guide - then the additional guide data could be "background" loaded at a slower rate (or on demand if the timeline view is loaded and the user begins to reach the end of the loaded data range).

Its entirely possible that's what's happening right now, but it seems like a good idea in my head :p
Reply
#11
Swifty, yes, there is room for improvement and your suggestion to only load like 3-6 h on startup and the rest in background seems like a good way to speed things up Also I see no need to block LiveTV because of not yet loaded timers or recordings - those could also be delayed. IIRC someone created a PR to give access to LiveTV functionality way earlier, but I think it had some issues in rare conditions.
Reply
#12
Glad to hear it could be a viable solution.

The startup time and EPG Grid performance are really the only things that punish the Pi when using XBMC (from my use at least) - I'm not sure why it would be such a resource intensive thing, but the EPG grid takes a good 10-15 seconds to load (every time you switch to it).

Hopefully as the PVR becomes more polished the experience for weaker clients will improve Smile
Reply
#13
the add-on must be doing something bad. hundreds of channel, several days of epg and recordings are loaded within seconds when using tvheadend on a pi, so that's one thing that needs fixing.
someone did a PR to make the pvr windows available while things are loading, but never finished it. it's on github, and can go in when completed and fixed.
opdenkamp / dushmaniac

xbmc-pvr [Eden-PVR builds] [now included in mainline XBMC, so no more source link here :)]
personal website: [link]

Found a problem with PVR? Report it on Trac, under "PVR - core components". Please attach the full debug log.

If you like my work, please consider donating to me and/or Team XBMC.
Reply
#14
would be good for marc.bau to run tvheadend setup on his Pi then, that would help to see whether it is related to any other network/hardware/tuner etc in his setup or it is our addon. Anyway until we have the requested logs we dont have any data or timings to go off. In the case of tvheadend client PVR addon, do you send every channel/EPG request to the server or do you cache stuff locally in the addon?

edit: it does seem that we have at least 2 addons mentioned above (pvr.wmc and mediaportal) which are taking considerable time to load all the EPG data on rpi's... Perhaps both addons could do something differently (that tvheadend is already doing?). Would be interesting for someone to try all 3 backends on the same hardware/network to compare and highlight any weaknesses in any, so they can be addressed
pvr.wmc TV addon and ServerWMC Backend Development Team
http://bit.ly/ServerWMC
Reply
#15
If it helps at all I can provide logs for a pi with the media portal addon, to compare with?
Unfortunately my tuner isn't supported in Linux so I can't try tvheadend.

@opdenkamp - out of interest how does the EPG grid perform in the setup you mentioned above? (mainly the load time each time you switch to the view)
Interested to know if this could be something addon related too, or if its just quite a task for the pi.
Reply

Logout Mark Read Team Forum Stats Members Help
LiveTV channels and EPG not cached0