2015-09-08, 22:11
The purpose of this topic is to discuss mechanisms to enable addons such as our addon 'Emby for Kodi' to replace the Kodi DB and scrapers as a back end.
History:
Emby for Kodi started out as an addon called XBMB3C which started as a fork from PleXBMC(using MediaBrower/Emby instead of Plex) in late 2013. The addon originally worked with the traditional 'ListItem' methodology - which worked but as we expanded to try to completely replicate the native Kodi DB speed and experience we hit a wall, particularly with large collections. Also, this methodology required custom skins - which became a lot of work in themselves.
Earlier this year, marcelveldt had the idea to try using .strm files and JSON-RPC to build a Kodi DB based on the Emby DB. This created a virtual filesystem with the .strm files, which Kodi then scanned in. The meta-data was then back-filled using JSON-RPC where possible. In this setup, the Kodi DB is really a local cache of the Emby DB.
Over time, due to performance reasons and missing JSON-RPC access, this was replaced with direct DB creation/updates. At the time, we did some searches and saw similar requests for speed ups and more access - and so we didn't report our issues. The intent here is to rectify that.
Issues:
1. Creating the virtual file system containing strm and scraping the information in proved to be quite slow. It was okay for a movie collection, barely tolerable for TV and intolerable for music. The process to build my db (as an example) is currently 3 minutes on my main PC and 15 minutes on my Pi. The strm version took 45 minutes and 8 hours respectively.
2. The JSON-RPC itself is considerably slower than direct DB access. At startup we do a comparison of the local DB to the Emby DB - and again this process is considerably slower. In my example DB, 30 seconds vs. ~4 minutes on a fast machine, and 8 minutes to 45 minutes on a Pi.
3. When the JSON-RPC is running the startup comparison above, there are considerable GUI glitches. The Kodi interface becomes slow and the screen flickers. This also occurs at end of playback when we would do playstate updates.
4. The JSON-RPC does not have access to some critical fields. We need to be able to set any and every property. Some missing controls are:
Way Forward:
Everyone on the team is a huge Kodi fan and wants to work with the Kodi team as much as possible.
Our goal is still to completely replace the Kodi scraper and all the artwork scripts - we are willing help with JSON-RPC enhancement, and/or other ways to achieve this (like the UPNP-based DB discussion).
We would like to ask that a Kodi dev sets up Emby and Emby for Kodi to see what we are doing and really understand the extent to which we are taking over DB responsibility. Emby is now easy to install for most Linux distributions. While we know we are breaking the rules - the thing works really, really well.
Regards,
marcelveldt, null_pointer, angelblue05, im85288, xnappo, jurmb84
History:
Emby for Kodi started out as an addon called XBMB3C which started as a fork from PleXBMC(using MediaBrower/Emby instead of Plex) in late 2013. The addon originally worked with the traditional 'ListItem' methodology - which worked but as we expanded to try to completely replicate the native Kodi DB speed and experience we hit a wall, particularly with large collections. Also, this methodology required custom skins - which became a lot of work in themselves.
Earlier this year, marcelveldt had the idea to try using .strm files and JSON-RPC to build a Kodi DB based on the Emby DB. This created a virtual filesystem with the .strm files, which Kodi then scanned in. The meta-data was then back-filled using JSON-RPC where possible. In this setup, the Kodi DB is really a local cache of the Emby DB.
Over time, due to performance reasons and missing JSON-RPC access, this was replaced with direct DB creation/updates. At the time, we did some searches and saw similar requests for speed ups and more access - and so we didn't report our issues. The intent here is to rectify that.
Issues:
1. Creating the virtual file system containing strm and scraping the information in proved to be quite slow. It was okay for a movie collection, barely tolerable for TV and intolerable for music. The process to build my db (as an example) is currently 3 minutes on my main PC and 15 minutes on my Pi. The strm version took 45 minutes and 8 hours respectively.
2. The JSON-RPC itself is considerably slower than direct DB access. At startup we do a comparison of the local DB to the Emby DB - and again this process is considerably slower. In my example DB, 30 seconds vs. ~4 minutes on a fast machine, and 8 minutes to 45 minutes on a Pi.
3. When the JSON-RPC is running the startup comparison above, there are considerable GUI glitches. The Kodi interface becomes slow and the screen flickers. This also occurs at end of playback when we would do playstate updates.
4. The JSON-RPC does not have access to some critical fields. We need to be able to set any and every property. Some missing controls are:
- Ability to add new items
- Full control over artwork
- Control over stream details
- Control over boxsets
Way Forward:
Everyone on the team is a huge Kodi fan and wants to work with the Kodi team as much as possible.
Our goal is still to completely replace the Kodi scraper and all the artwork scripts - we are willing help with JSON-RPC enhancement, and/or other ways to achieve this (like the UPNP-based DB discussion).
We would like to ask that a Kodi dev sets up Emby and Emby for Kodi to see what we are doing and really understand the extent to which we are taking over DB responsibility. Emby is now easy to install for most Linux distributions. While we know we are breaking the rules - the thing works really, really well.
Regards,
marcelveldt, null_pointer, angelblue05, im85288, xnappo, jurmb84
