Option to have XBMC use more available memory resources [NOT XBOX Platform]

This thread is for the discussion of a feature request before/if it is added to trac. This is relevant to the non XBOX versions.


When you install XBMC it runs like a normal program. In many cases that is fine but what if you have an HTPC dedicated to XBMC?

I run a fairly large video library (26MB on disc) and on win32 XBMC never uses more than 160MB of RAM. This machine is dedicated to XBMC and has 8GB of memory but currently there is no way to make XBMC break it confines and start hogging memory.

There are lots of areas that would benefit from further memory use but the two that jump out are caching of thumbnails (for slicker folder and library browsing) and caching of older video frames (for slicker RW and step back).

I cant comment this code wise but in theory it makes a lot of sense to me.

Comments/ideas?

See feature request:
http://trac.xbmc.org/ticket/4640
and feature request:
http://trac.xbmc.org/ticket/4641

Both of those tracs are very interesting indeed. However neither touch on non video caching and in fact both focus on ways to maximise memory usage efficiency specifically for video playback.

To augment these, some thought should be given to caching all elements in UserData and indeed other data such as pictures and music. My userdata for instance contains just over 20,000 tbn files and whilst no view within XBMC would ever load all these the natural process of using the library will load a sizeable portion of them. I propose a method whereby userdata that is loaded once resides in ram for the duration of the session or is reclaimed if actually required to due a memory shortage (this is the crux of my idea). Drilling down and then back up in library view could therefore be optimised and disk access and more importantly responsiveness to the user improved.

I also believe most of the discussion re video cache are concerned around pre-emptive caching of data required for future playback, be it a few frames or a few seconds of video that are about to be played. The video that has just been played is less valuable to be cached but still valuable enough to not be dropped unless it needs to be.

I can think of 2 easy gains:

1. Have a (long) timeout on texture unloading, rather than instant unloading.

2. Have a better thumbnail cache.

2 applies without needing much more memory - basically saves hitting the disk all the time to check for existence.

A patch would be most welcome. See guilib/TextureManager.cpp, and look for ReleaseTexture(). Essentially you want to have an additional variable to ensure it doesn't unload, then an additional function to unload any expired textures. See how the xbmc/GUILargeTextureManager.cpp does it for instance.

Cheers,
Jonathan

From what i understand of what you are saying this seems like an excellent idea. Should I trac it as a feature request for 9.1? There is absolutely no way I could create a patch myself.

fyi i am xe' on IRC and pretty much there all the time should any real time discussion be needed.

Why not create a ram drive and store your UserData folder in there? Or maybe even just the whole XBMC folder should be in a RAM drive. That way the app, any dlls it loads etc are all already in ram..

would this improve the speeds at which the fanart loads up?

EvilDude Wrote:Why not create a ram drive and store your UserData folder in there? Or maybe even just the whole XBMC folder should be in a RAM drive. That way the app, any dlls it loads etc are all already in ram..

I tried the latter option on linux, move all XBMC to ram and the speedup isnt that great actually. Actually more of a waste of it, I thought it would do more but

Anyway, the databases and covers and such (mostly everything in UserData) would probably gain alot from this but it would add on start and shutdown time, if not designed properly.

Another gain would be to pre-load images that might be needed, thinking mainly on Home of most skins, they have lots of big images. long release would work here aswell.

I thought about the ramdrive myself but my userdata folder is 777MB so that would represent to much of a delay on boot up. Also even if i could make it work i then have to deal with cleanly copying the changed files back to disk on shutdown which would be a kludge and even if it did work would make XBMC super slow to shut. In general people can accept slower boot times but cannot accept slow shut times and tend to kill program either manually or an OS shutdown. This approach also doesn't scale well to extrmelly large collections.

I cannot see the advantage of pre-load other than tricking users into thinking some views load faster at the expense of slower booting. If this could be a background process it would likely work but it seems like the wrong approach to me.

I am not looking for faster initial load times of XBMC or the views but rather faster load times on subsequent visits to pages you have already loaded content for/been to.

This is why I am saying keep things cached rather than drop them where the possibility exist.

xexe Wrote:I thought about the ramdrive myself but my userdata folder is 777MB so that would represent to much of a delay on boot up. Also even if i could make it work i then have to deal with cleanly copying the changed files back to disk on shutdown which would be a kludge and even if it did work would make XBMC super slow to shut. In general people can accept slower boot times but cannot accept slow shut times and tend to kill program either manually or an OS shutdown. This approach also doesn't scale well to extrmelly large collections.

I cannot see the advantage of pre-load other than tricking users into thinking some views load faster at the expense of slower booting. If this could be a background process it would likely work but it seems like the wrong approach to me.

I am not looking for faster initial load times of XBMC or the views but rather faster load times on subsequent visits to pages you have already loaded content for/been to.

This is why I am saying keep things cached rather than drop them where the possibility exist.

Are you sure you know where the delays you experience are? I have a large database for music and video and only really experience delays in the music library such as opening or returning to large library views which JMarshall has traced to in memory parsing/processing of the data for the selected view after its read from the database.
For video I dont really have any noticable problems but then the number of videos will always be far less than music tracks anyway