External database updater?
#1
Hi.. is there any tool that can manipulate kodi database directly? updating via kodi /JSON has 2 issues for me :

- Kodi needs to be open on server, my diskless setup does not allow this, long story short opening kodi will lock the database and diskless client wont be able to access it..
- I frequently get "unable to connect to server" due to thetvdb's unreliable uptime... (is there no known way to "auto-yes" the message?)

For now i'm using a dirty workaroung by symlink the userdata and syncronizing it using syncovery.. but it still locks up if a client boots then the server updates kodi..

I've seen some tools mentioned in wiki but they dont seem to offer this feature
Reply
#2
- re: db lockup > suggest not using sqlite (a db file) do a msSQL db running elsewhere that all clients can reach. http://kodi.wiki/view/MySQL
- re: ignoring errors > this is an advanced setting > http://kodi.wiki/view/advancedsettings.xml#videoscanner
rPi 2&3 | android phones | fireHD8 | linux | win10 + NFS NAS w/ mySQL + props to...
libreElecyatse, titan, AELflexGet, context.manageTags (a zosky original)
Reply
#3
(2017-03-08, 11:08)zosky Wrote: - re: db lockup > suggest not using sqlite (a db file) do a msSQL db running elsewhere that all clients can reach. http://kodi.wiki/view/MySQL
- re: ignoring errors > this is an advanced setting > http://kodi.wiki/view/advancedsettings.xml#videoscanner

Kudos for ignore error tips didn't know that..

Would that mysql setup work better than my diskless setup in terms of perfrormance?
I have over 25 clients btw

Because i'm getting near local kodi performance by PXE boot the kodi database in forms of vhd images..
Reply
#4
not sure how it would scale to 25. i have 5 clients, and the db on recycled hardware - a core 2 duo - mysql is 3rd highest CPU hog (after flexGet and transmission) but its still plenty responsive when im browsing the library

for sure it would solve the db locking issue, but it might introduce a different one if the machine hosting your db cant keep up with the clients and has to start queuing queries... then you'll need to look at mySql optimisation. increasing its ram, caching common queries and such
rPi 2&3 | android phones | fireHD8 | linux | win10 + NFS NAS w/ mySQL + props to...
libreElecyatse, titan, AELflexGet, context.manageTags (a zosky original)
Reply
#5
If you want to centralize your content and the database data MySQL is the way to go. You would need to spec the server based on your needs but I have it running in a vm with only 5 clients and barely see anything happening. I have a tonne of other tasks running on that vm server as well.

Also it would be best to have a Kodi instance also running on that server that would update the MySQL database whenever new data is added. It will basically sit quietly minimized and only kick in when it scans for new data. Then all the clients would just use that database and sync things such as watched status and cover art directly from it. Using the MySQL database will not create lockups.

The other way would be that each of your instances use their own database and they all scan for new content at start up so that they are all up to date whenever they start up. This way will however make things slower when they start and also not sync things such as watched status (if that is important).

One more option (but not as pretty) is just to have the file paths as sources in Kodi and navigate them via Video's -> Files. You won't have the pretty cover art as it doesn't import all the data into the database but at least you don't have to deal with scanning and syncing data. Just thought I would mention this as everyone's needs may be different. I use this method for "other videos" that aren't movies or tv shows.

Hopefully that gives you some ideas.
Reply
#6
(2017-03-08, 20:51)zosky Wrote: not sure how it would scale to 25. i have 5 clients, and the db on recycled hardware - a core 2 duo - mysql is 3rd highest CPU hog (after flexGet and transmission) but its still plenty responsive when im browsing the library

for sure it would solve the db locking issue, but it might introduce a different one if the machine hosting your db cant keep up with the clients and has to start queuing queries... then you'll need to look at mySql optimisation. increasing its ram, caching common queries and such

Hmm still not convinced MySQL could be faster than pxe boot approach..

(2017-03-08, 21:09)GavinCampbell Wrote: If you want to centralize your content and the database data MySQL is the way to go. You would need to spec the server based on your needs but I have it running in a vm with only 5 clients and barely see anything happening. I have a tonne of other tasks running on that vm server as well.

Also it would be best to have a Kodi instance also running on that server that would update the MySQL database whenever new data is added. It will basically sit quietly minimized and only kick in when it scans for new data. Then all the clients would just use that database and sync things such as watched status and cover art directly from it. Using the MySQL database will not create lockups.

The other way would be that each of your instances use their own database and they all scan for new content at start up so that they are all up to date whenever they start up. This way will however make things slower when they start and also not sync things such as watched status (if that is important).

One more option (but not as pretty) is just to have the file paths as sources in Kodi and navigate them via Video's -> Files. You won't have the pretty cover art as it doesn't import all the data into the database but at least you don't have to deal with scanning and syncing data. Just thought I would mention this as everyone's needs may be different. I use this method for "other videos" that aren't movies or tv shows.

Hopefully that gives you some ideas.

Are you using esxi or similar approach?
Why not boot the db as part of the VM?

I'm using ccboot which is a PXE boot software.
I'm not quite sure the difference with your method,,
but ccboot has what it's called "gamedisk" which is essentially upgradable content for clients..
gamedisk content can be upgraded in the server, and the clients have to reboot once to get the updates.

So i'm doing this with Kodi, by symlink'ing the kodi db in the server OS with db for client gamedisk.
So Server updates Kodi > symlink'ed Database in gamedisk updated > client reboots and get update

So essentially all kodi database in client is loaded during booting process, and then cached in diskless server..
hence i'm getting a very smooth local like Kodi performance even with 25 clients

So i guess the difference is
PXE boot = db loaded during OS boot
MySQL = db loaded when Kodi is open after OS boot

Not sure mysql could match this because some time ago i tested it with a few users and there were some lags..
Reply
#7
(2017-03-09, 11:10)denywinarto Wrote: Are you using esxi or similar approach?
Why not boot the db as part of the VM?

I'm using ccboot which is a PXE boot software.
I'm not quite sure the difference with your method,,
but ccboot has what it's called "gamedisk" which is essentially upgradable content for clients..
gamedisk content can be upgraded in the server, and the clients have to reboot once to get the updates.

So i'm doing this with Kodi, by symlink'ing the kodi db in the server OS with db for client gamedisk.
So Server updates Kodi > symlink'ed Database in gamedisk updated > client reboots and get update

So essentially all kodi database in client is loaded during booting process, and then cached in diskless server..
hence i'm getting a very smooth local like Kodi performance even with 25 clients

So i guess the difference is
PXE boot = db loaded during OS boot
MySQL = db loaded when Kodi is open after OS boot

Not sure mysql could match this because some time ago i tested it with a few users and there were some lags..

Ahh.. I get what you are doing. You are basically using the Kodi internal database and copying it everytime the client reboots. Doing it this way is kind of a hack and yes can cause issues depending on how the gamedisk operates. Its similiar to me taking a copy of the MySQL database to each client. With MySQL your not copying the database, you are telling kodi where the server is so it can ask MySQL for the data and update it.

When the client boots up is it just working off of a copy (that gets discarded) of it or do changes made from the client get written back to the server "master copy".

The proper way to share the data between multiple clients is using mysql. You will have a bit of a lag because the client needs to query the database and get the information as you navigate the menu's but its not that bad in my environment. You also get the benefit of having watched status between all of your clients. You can watch a show, stop it and go in another room and continue where you left off. You can also have one client be the master and it will update the MySQL database and the other clients will see the updates in pretty much real time.
Reply

Logout Mark Read Team Forum Stats Members Help
External database updater?0