"Trickle Caching" for HD Content Streaming

These days, it's no longer unusual to have 1080P content streamed over the internet, and there are now multiple XBMC plugins that access this content. There is however a problems, in that these streams saturate memory caches. The reason that's a problem, is when the cache is full you can't download anymore and downloading is paused. The reason that's a problem is that there's an latency involved from taking a paused download back to full use of available bandwidth.

This is the one of the reasons people see problems with 1080P streamed content in XBMC despite having internet connections that should handle it. Because XBMC grabs as much as it can right away, it fills the cache up, then stops downloading for a bit, then tries to download again when the cache dips down. If the latency between paused download and full use of bandwidth is too great, then a buffer underrun will occur. This is what causes the skipping in and out of apparently speedy buffering.

This is basically a "Greedy Scheduling" problem. Ideally, you want to always be downloading, so there's no gap between paused download and best use of bandwidth. But if you take everything as fast as it can arrive, then you'll fill the buffer, causing a buffer overrun forcing you to pause downloading. So you're alternating between buffer overruns and buffer underruns.

The way to avoid this is actually pretty simple. Just don't download it all at the highest speed possible, and instead always have a trickle of content coming in to try and constantly be filling the buffer. The classical explanation is filling a bucket with a hole, you don't want to have the tap on full and over flow the bucket, and you don't want to keep turning it on and off because then the bucket will run dry.

The network stack actually does have a lot of clever ways to throttle back downloads, and reduce your bandwidth being used. And the best part is that most stacks do this without having to make any special system calls at all. Just limit how many read() calls are made in any period of time, and the network stack will try to throttle the download to the same rate.

So by rate limiting the number of read() calls, you can keep the buffer in check. Work out a 'sweet spot' in the buffer. Over that, start decreasing the number of read() calls. Under it, start increasing the number of read() calls. Theoretically, you can avoid ever overrunning, and if enough bandwidth is there you should also prevent underruns.

XBMC already does this.

In that case, the cache sweet spot needs adjusting, because it's acting like it doesn't. Perhaps there's simply too little cache to handle 1080p bitrates? Remember we're talking in terms of 3-4mbps streams, so really there needs to be 16mb caches to handle more than few seconds of stream. And that's the lower end. Remember that the ATV3 upgraded their system memory from 128mb to 256mb to be able to handle 1080p streaming, and the bulk of that is for the in-memory cache.

If XBMC has issues with large in-memory caches, perhaps it's really time to start looking at on-disk caching again?

What are you trying to accomplish?

Er... I do not understand the question? I'm suggesting improvement is needed in XBMC's network caching, because it's running up against delivery issues for 1080p bitrates. Do I need to state any more motive than that?

ATV2's had 256MBs of ram, ATV3's have 512MBs of ram, the change has nothing to do with 'streaming' 1080p. It has to do with a common SoC that can be used on iPads.

References I read at the moment suggest that the memory cache has issues with being set above 8mb? (Three times as much memory is allocated?)

What are the low-watermark and high-watermark set at?

---

Sorry, I confused my models when typing.

But... the ATV3 does not share a common SoC with the new iPad. (ATV3 - A5, iPad generation 3 - A5X) It actually only shares components with the Rev2 iPad 2, and only the dieshrink A5. But I'm not sure why you're arguing this, surely it's a given that higher bitrate content needs revision of assumptions about memory caches?

(2012-05-07 22:01)barberio Wrote:  Er... I do not understand the question? I'm suggesting improvement is needed in XBMC's network caching, because it's running up against delivery issues for 1080p bitrates. Do I need to state any more motive than that?

I'm all up for improvements, but I can't improve anything if I don't know what the problem is in the first case.

(2012-05-07 22:09)barberio Wrote:  References I read at the moment suggest that the memory cache has issues with being set above 8mb? (Three times as much memory is allocated?)

What are the low-watermark and high-watermark set at?

---

Sorry, I confused my models when typing.

But... the ATV3 does not share a common SoC with the new iPad. (ATV3 - A5, iPad generation 3 - A5X) It actually only shares components with the Rev2 iPad 2, and only the dieshrink A5. But I'm not sure why you're arguing this, surely it's a given that higher bitrate content needs revision of assumptions about memory caches?

And where do you think the sram is located on an A5 Hint, it's on chip, the A5 is a die stack.

Well, I do notice there seems to be issues that xbmc's default cache stategy results in 'bouncing' in and out of cache for streams when there is ample bandwidth available. This is pronounced when it's a source that had high bandwidth but high latency, which is exactly what I would expect for hitting buffer overruns too much.

Can I check, is the default stream cache 5mb as the wiki documentation suggests? I've just changed that to 64mb in advancedsettings.xml and tested with a source that was showing the problem, and it seems to no longer get the cache 'bouncing', suggesting that the current default is too low for usable cache, or the high watermark is too high. Is there a hard-limit to this cache size, and what is it?

---

(2012-05-07 22:30)davilla Wrote:  And where do you think the sram is located on an A5 Hint, it's on chip, the A5 is a die stack.

This really is nothing to do with what we're discussing. But again, the new iPad that they're producing in huge numbers doesn't use the new die-shrink A5 in the ATV3. It uses the A5X which is an entirely different component. The A5 dieshrink is being used in the low volume rev.2 iPad2, but that's more because they're not making the original A5s any more.