XBMC not optimized for large music libraries/low powered systems?
#1
Many of us here use small, low powered systems to run XBMC. In theory this makes a lot of sense for an always on type of media appliance. For the most part I think these low powered devices (Atom based) work fine for libraries that are not too large. The problem I have is with my music library. I have over 30K songs. Accessing my music library takes quit a bit of time. I have done everything I can to help speeds things up by adding more memory, an SSD, and enabling DDS. Even with these changes there is still quite a lag. Short of getting a more powerful PC, I am not sure what else I can do.

I wonder if XBMC developers are considering this issue as well. People's digital libraries will only get larger over time. No one wants to put a full blown PC by their media center. Perhaps something can be done to help make access to large media libraries quicker with lower powered pcs.
#2
Here are a few things I'd look at to improve speed:

local vs network file storage
network filesharing protocol (SMB vs NFS vs FTP)
network speed (wireless, gigabit, 100mbit)
Database used (sqlite vs mysql)
Media File organization (lots of music files in one directory vs organized by genre, artist, year, etc)

I have just over 39k songs in my library and using an zotac mag atom based nettop on one of my HTPCs. All files are on a centralized fileserver, using SMB file shares. All network interfaces and switches are gigabit. The screens that take the longest to load (1-2 sec) are:

Root file list (in file not library mode)
Artist list

Overall I'm satisfied with the speed in XBMC's library.

I have seen anecdotal reports that NFS performs better with large number of file or folder nodes than SMB, but I haven't had a chance to try it myself.

I suspect that performance could be improved by switching your xbmc database from a local sqlite to networked mysql, but YMMV.

Lastly - what's your setting under Settings -> Music -> File Lists, "Search for thumbnails on remote shares" set to? If on, it's going to make a network request while browsing artists and albums.
#3
1. If you're using the library, the filesystem is irrelevant.

2. To speed up some lists, ensure you're not using mysql.

3. There are possibly some improvements to be made in terms of the queries as some fields aren't indexed when they should be.

4. What is slow? Obviously "songs" will be as you're retrieving 30k items. Anything else shouldn't be too bad. Plenty of folk had libraries of this size on xbox.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
#4
Thank you for the replies.

I would be very satisfied with a 1-2 sec load time!

Here is my setup:
Media stored on a Thecus N4100Pro NAS hardwired to router
"Server" PC - Zotac ND-22 (Celeron SU-2300 processor) hardwired to router, mysql, shared Thumbnails folder via symbolic link
Client PC - Acer 3700 (atom 525), dedicated wireless N

On my Zotac which has the mysql and Thumbnails folder stored locally, accessing the Artist screen takes about 5-7 secs. On my Acer which accesses the network wirelessly, it takes about 45 secs! Before I shared the Thumbnails folder, it would take the Acer about 10 secs. My wireless network is able to stream 1080p movies without any issues.

I wonder if I put mysql and shared the Thumbnails on a faster PC if things would improve? Is the ZOTAC just not fast enough to serve up all the data?

When XBMC tries to access the Artist screen, does it read/load up all the images and data? If so, would it make more sense to just load up blocks of data based on last artist visited? For example, If the last artist or position I accessed was Sade, the next time I open the Artist screen, XBMC would load up 50-100 artists around the "vicinity" of Sade. As I browse through my Artist library, XBMC would load up more blocks of data as required. Of course there would a small lag but it should be less than having to wait for the entire library to load.
#5
Quote:mysql, shared Thumbnails folder via symbolic link

There's your problem. Ofcourse it's bloody slow.

First off, fix the indexes - there's a bunch of posts about this, so search the forums.

Secondly, the thumbnails being non-local *may* slow it down (and may slow it down tremendously) - it won't do so on all views, but may do so on some. The speed of the PC they're on will be basically irrelevant. This will be taken care of when the music thumbs move to the texturecache - hopefully before Eden. For now, your only option is keeping them locally if you want reasonable speed.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
#6
jmarshall Wrote:There's your problem. Ofcourse it's bloody slow.

First off, fix the indexes - there's a bunch of posts about this, so search the forums.

Secondly, the thumbnails being non-local *may* slow it down (and may slow it down tremendously) - it won't do so on all views, but may do so on some. The speed of the PC they're on will be basically irrelevant. This will be taken care of when the music thumbs move to the texturecache - hopefully before Eden. For now, your only option is keeping them locally if you want reasonable speed.

Cheers,
Jonathan

Thank you Jonathan. I understand and expected the sharing the Thumbnails folder to slow things down. I just didnt expect that much. However, even when the Thumbnails was stored locally it would take 5-10 secs to load the Artist screen.

Looking forward to the texturecache. Will this still allow for sharing the Thumbnails folder between multiple XBMC clients?
#7
The 5-10 seconds is due to mysql's lack if indicies.

Search the forums here and you'll find the indicies that need adding and that'll speed things up nicely.

It would probably be useful if you could track them down and then add a trac ticket regarding them so that we can add the required indicies to HEAD so they're there for everyone by default.

And no, the texture cache does not allow or require sharing the thumbnails folder. Rather, it has a local thumbnail cache on each client (i.e. fast loading) which is automatically updated, as the URI as to where to find the original thumb files are in the already shared media libraries.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
#8
Here's a post with the indexes. I tried them and they made a noticeable improvement in the speed of navigating the music library when using a mysql db.

http://forum.xbmc.org/showpost.php?p=728...tcount=639

The two listed for the Movie and TV section didn't have any noticeable affect. Hopefully someone will figure out the best indexes for those sections too.
#9
Thanks everyone. I have already added the indices to mysql, yet there is still a noticeable delay.
#10
I have set up Eden Beta 1 with a shared mysql db and the performance on the artist listing is not that great (30 seconds to display), album is a bit better (10 seconds). I do have a large library 35000 songs but I was wondering if any of the index creation and tips to speed up the db made it inside Eden ?
#11
tboooe Wrote:Thank you Jonathan. I understand and expected the sharing the Thumbnails folder to slow things down. I just didnt expect that much. However, even when the Thumbnails was stored locally it would take 5-10 secs to load the Artist screen.

Looking forward to the texturecache. Will this still allow for sharing the Thumbnails folder between multiple XBMC clients?

What i did to speed this up on my slow boxes is use local storage synced by dropbox, I symlinked the libraries/thumbs to my dropbox folder and have so far 3 boxes in sync (one is in another house actually connected by a dedicated point to point wireless link) it's local storage except that it's synced up Big Grin, pretty fast, the free dropbox account is more than enough for me so far
#12
I'm using Eden beta, have 3 clients, and a windows home server, with mysql.

I found that storing thumbnails on the server really slowed things down for me. I had done it using the <pathsubstitution> tag in the advancedsettings.xml file, and while it did work, the lag when loading up any listing was driving me crazy (5-10 seconds simply to list the episodes in a single tv show)

After studying the log file, the database was finished with the query in about 0.1 seconds, and then the thumbnails were taking about 0.3 seconds each.

My ultimate solution was to keep the thumbnails on the local clients, and then each night I run a scheduled task on the server that synchronizes them all between the machines. After that xbmc ran like a charm. Very very snappy response times.
#13
fbarwell Wrote:My ultimate solution was to keep the thumbnails on the local clients, and then each night I run a scheduled task on the server that synchronizes them all between the machines. After that xbmc ran like a charm. Very very snappy response times.

How did you accomplish this? If you don't mind me asking.
#14
(2012-01-26, 20:02)BULLIT Wrote:
fbarwell Wrote:My ultimate solution was to keep the thumbnails on the local clients, and then each night I run a scheduled task on the server that synchronizes them all between the machines. After that xbmc ran like a charm. Very very snappy response times.

How did you accomplish this? If you don't mind me asking.

I'm also curious to know how this was done, as I would like to have them fully sync'd, but can't stand any delays and am thinking this is probably the most ideal method of doing this.

As an alternative, would it be possible to have xbmc cache the thumbnails the first time it fetches them, and in future instances only check to see if they've changed? I imagine things would slow down exponentially more if it's trying to fetch fanart as well as the thumbnails?
XBMCbuntu {Frodo} - Ubuntu 12.04 (Main):
Intel Q6600 [2.4GHz Quad Core], 4GB DDR3, nVidia GT-210, Antec Fusion Case (iMon LCD), PS3 BD Remote

XBMC Live {Frodo} - Ubuntu 10.04 (Bedroom): Intel Atom D510 [1.66GHz Dual Core], ION Graphics, 2GB DDR2, PS3 BD Remote
#15
(2012-01-26, 20:02)BULLIT Wrote: How did you accomplish this? If you don't mind me asking.

Here's a sample batch script for copying files using FTP.
http://www.howtogeek.com/50359/upload-fi...ch-script/

Logout Mark Read Team Forum Stats Members Help
XBMC not optimized for large music libraries/low powered systems?1