Issues with MySQL backend and OpenElec
#1
I have setup OpenElec on a raspberry PI with a MySQL backend on a Synology NAS. The media content and thumbnails are all stored on the NAS in different NFS shares - thumbnails are redirected using pathsubstitution. I'm able to access the media, perform content scraping and playback. I have two challenges that I'm hoping the community might be able to provide some assistance with:

1) Thumbnail generation is terribly slow when navigating the UI. Does anyone have any suggestions on how to improve this?

2) If I end up deploying additional devices, accessing the MySQL backend would work great, but how do I handle multiple users for items like bookmarks and watched list? I've read that there isn't really an easy way to accomplish this - it looks like most people create multiple databases for each profile, but having to scrape content multiple times is ridiculous especially if it is the raspberry PI taking care of this (no synology NAS solution to scrape content in the backend). I've also found that bakslash managed to hack the MySQL database to support multi-user logins (see here), but I haven't been able to get his script working. I'm also a little uneasy about hacking up the database as you'd likely lose the ability to upgrade seamlessly. Anyone have any suggestion on how to handle multiple users on multiple devices?

Thanks.
Reply
#2
Which version of XBMC are you using? (Gotham will help here Wink )And why path substititurion for thumbnails? It's better to store the thumbnails on the pi locally, this will reduce the time needed for getting the data (the network interface isn't that quick). THe path substitution is not needed for sharing the thumbnails and (I guess) pre frodo builds.

I don't have an idea about the profile question.
Reply
#3
Why are you using path substitution for the thumbnails - are you really that short of space on the Pi? Generally speaking path substitution is a bad idea, unless you really don't have sufficient storage (such is the case with ATV etc.).

1) See sig.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#4
I'm using the 12.2 Frodo (OpenELEC Stable - Raspberry Pi ARM Version:3.2.4). Most guides that I read online regarding using a MySQL back-end included using path substitution, but I guess most are written pre-frodo. I assume I can simply remote path substitution from the advancedsettings.xml and reboot (I'll probably end up doing this before a response :p )? Just for my own understanding regarding the history, caching of thumbnails locally only began with Frodo? I would have figured that loading thumbnails over the network would perform faster than loading them from the SD card (100 Mbps Ethernet vs 80 Mbps of class 10), but it appears to be not the case.

@MilhouseVH I've actually found your texture cache maintenance script from your signature in other threads, but I'm not entirely certain on how to implement it. Is this something that can be automated with a cron job or is it meant to handle one off cases where we discover a corrupt image?

Yes, I am still lost on how to handle multiple users and multiple devices. It looks like the only way at this point is multiple databases. I've seen a lot of random asks about this and for server-based scraping (i.e. xbmc based server). It would be great to see this developed in the future.

On a side note, I'm actually surprised the raspberry pi only has 10/100 Ethernet. I just looked it up writing this post as I assumed it was Gigabit. Shocking that it handles large 1080p files so smoothly for video playback.

I couldn't wait...I removed path substitution and rebooted. Thumbnail generation is still terribly slow and it appears to only do it when hovering over the item for the first time (if at all). Is there anyway to have all the thumbnails generated without browsing? Would this occur from re-scraping?

This is also the other reason I used path substitution as I was using a much faster computer to perform the scraping and therefore generating the database entries and thumbnails.
Reply
#5
(2014-04-01, 02:12)osnabrugge Wrote: I would have figured that loading thumbnails over the network would perform faster than loading them from the SD card (100 Mbps Ethernet vs 80 Mbps of class 10), but it appears to be not the case.
Reading from SD card at 20MB/s (bytes) is faster than a 100Mb/s (bit) network.

(2014-04-01, 02:12)osnabrugge Wrote: @MilhouseVH I've actually found your texture cache maintenance script from your signature in other threads, but I'm not entirely certain on how to implement it. Is this something that can be automated with a cron job or is it meant to handle one off cases where we discover a corrupt image?

It can be automated with a cron job, or run from the command line.

(2014-04-01, 02:12)osnabrugge Wrote: On a side note, I'm actually surprised the raspberry pi only has 10/100 Ethernet. I just looked it up writing this post as I assumed it was Gigabit. Shocking that it handles large 1080p files so smoothly for video playback.
Most 1080p files are well under 100Mbit/s, in fact most would be closer to 35Mbit/s, making a GigE connection overkill.

(2014-04-01, 02:12)osnabrugge Wrote: I couldn't wait...I removed path substitution and rebooted. Thumbnail generation is still terribly slow and it appears to only do it when hovering over the item for the first time (if at all). Is there anyway to have all the thumbnails generated without browsing? Would this occur from re-scraping?

That's what the script in my sig is for (at least, that's how it started out - does a bit more than that now). Use it to pre-load the texture cache on the Pi (or any other XBMC client).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#6
Your texturecache script works like a charm Smile It would be nice to see this included officially within an openelec / xbmc release. I decided to create multiple profiles using multiple databases. I hope that one day the architecture can be driven to have a more headless operation that supports scraping on automatically on the server with support for a multiuser database. These raspberry pi devices have great graphics performance, but aren't really designed to handle the CPU tasks of scraping media.
Reply
#7
actually, I am getting two issues related to your script that perhaps you know of a quick fix?

1) It appears that the script only generates thumbnails for the master profile. How do I generate thumbnails for additional profiles?
2) Sometimes I receive an error while executing the script:

FATAL: The task you wish to perform requires that the web server is
enabled and running on the XBMC system you wish to connect.

A connection cannot be established to the following webserver:
localhost:80

Check settings in properties file texturecache.cfg

I have logging enabled, but the only message I see is regarding a JSONRPC.Ping, but no failure. http://sprunge.us/hfYT
Reply
#8
try 8080
Reply
#9
I have set the webserver to port 80. I found the issue which is that the script won't connect to the webserver to interact with the database unless you are logged into one of the profiles on xbmc. I discovered this when attempting to manually post process via sickbeard which would crash every time it tried to send an xbmc update command. I'll just have to manually update xbmc, because I can't guarantee that someone will be logged in - and what do I do for other profiles?

Once again, this is another strike against this platform being anything designed to handle multiple users. Hands down, this is the best UI and HTPC experience out there, but it's disappointing that it's really designed for a single user environment. I really hope that one day we may see some development for this as I know I'm not alone.
Reply
#10
Profiles are a bit of a GUI hack, I guess, and it's a hack that falls apart when you start involving JSON which has no specific profile support, so the JSON requests will only operate on whatever profile is active at the time the JSON request is made. While this makes some sense it also makes it impossible to automate the maintenance of all profiles as it's not feasible to change profiles with JSON as this would change the profile used by the GUI, which would result in chaos for anyone using the system at the same time!

I suspect it would involve a lot of work to allow JSON to operate against different profiles to that currently being used by the GUI.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#11
Do you have any suggestions on how to handle multiple users? Right now, I've really hacked things up with multiple databases, multiple profiles to their respective database on each device. I've setup a single machine with watchdog logged onto to both profiles (2 separate xmbc instances). The only thing I'm lacking is getting your script to execute on each profile as per not being logged into the device.

Obviously, it would be far more elegant to perform the scraping directly on the NAS, a single database and the ability to automatically download textures, but I understand that the core architecture was never designed for this. I'm struggling to get this working, because I don't want to ditch the XBMC UI. Rasplex is somewhat closer to the solution, but the interface is not nearly as good.
Reply
#12
(2014-04-17, 02:10)osnabrugge Wrote: Do you have any suggestions on how to handle multiple users?

Not really, since XBMC and JSON both expect a profile to be active ("logged in") and since the script uses JSON it suffers from this same limitation.

There appears to be no way to activate a profile using JSON, so I can't think of a solution other than manually switching profile before running the script, which obviously isn't ideal. It would be great if it were possible to specify the profile on the JSON request, but that's not currently supported and I don't recall seeing any plans to implement such a feature.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#13
Although it isn't the answer I'm looking for, I do appreciate the response. Is there not significant demand from the user community for better multi-user support?
Reply
#14
I have an instance of XBMC running headless on my unRAID box.

I go to the settings webpage, and then have it update the library for me.

Its a plugin for unRAID: XBMC Standalone Library Updater for UnRAID
Reply
#15
I have a Synology NAS which doesn't have any headless XMBC support. My only other device lying around is shared for work and runs Windows Server 2012 R2 for Hyper-V. From what I've read, you cannot use unRAID as a VM as the Hyper-V network card isn't recognized. You have made me think of running a Linux VM as I've seen some forum activity around a headless XMBC Linux client. Do you know if you can perform updates to multiple database/profiles from your unRAID plugin (it's likely developed from the same forum activity that I've been reading)?
Reply

Logout Mark Read Team Forum Stats Members Help
Issues with MySQL backend and OpenElec0