Easily Sync Multiple XBMC Machines to 1 Library

hippojay · 2011-09-10 00:13

Hyram Wrote:Is there an option to force XBMC to not use its internal library, and keep all of its metadata alongside the media instead?

If not, there should be. It'd simplify things no end!

It would slow down functionality though, as without an internal library you'd need to cycle and read through those files. Unless I've misunderstood your idea?

Hyram · 2011-09-10 06:19

No, you haven't misunderstood. I feel that with most rendering duties being shunted off to the GPU (meaning there is CPU clock-cycles to spare by the metric tonne) and the prevalence of gigabit ethernet in the home LAN, any performance hit incurred by reading metadata directly instead of a database would be negligible.

YodaEXE · 2011-09-10 13:34

Hyram Wrote:No, you haven't misunderstood. I feel that with most rendering duties being shunted off to the GPU (meaning there is CPU clock-cycles to spare by the metric tonne) and the prevalence of gigabit ethernet in the home LAN, any performance hit incurred by reading metadata directly instead of a database would be negligible.

And you'd be mistaken. There would be a noticeable lag in the loading of things. Sure, it would likely only be a second or two, but that's a long time to be waiting for info to show up.

MacGyver · 2011-09-19 16:33

I too have tried to setup the SQL database method, and my lack of knowledge of Linux made me give up, the XBMC settings were trivial, but I had to rely on me setting up SQL on my Linux server, and I just wasn't devoted enough to solve my SQL issues to finish it.

Why can't XBMC continue to work the way it does, using the local *.db file, but have a setting that when turned on and pointed to a .xml file on a SMB share, it will compare it's local .db to the remote .xml, and then sync their content together. Obviously XBMC should make sure the remote file is not locked (in use by another XBMC) and if so cue the sync for later (each entry in the XML would need a timestamp for the changes made, so one XBMC doesn't undo the others). For that matter, the remote .xml could contain the path to where the remote thumbnail repository is located, and when the local database is syncing with it's remote .xml, it could pull/push the artwork too.

Having an non-local database is slow, so this seems like a "best of both worlds" solution, and makes each branch responsible for it's implementation. It would just be a new task that runs, after a reboot (after the initial "Update"), and before a proper shutdown (and maybe check to see if it's changed at various random times).
It would just require a share with read/write to put the .xml and artwork folder in from the end user, and two settings changes in the XBMC GUI. Hell I could even set it up if it was like that.

The way I imagine it is when a XBMC came online, it would have to scan the .xml for differences between it and it's local .db, then add the changes to it's local .db, then scan the media shares for changes, then update it's .db with those, then update the .xml to match.

Now because I have close to no programming experience, and no idea how the database updates work in XBMC, I'm sure there are about 50 reasons why this can't work.

Thanks for all the work you Devs do, I know enough to know that you guys are awesome.

HwyXingFrog · 2011-09-20 04:41

Even just having it as an option in the interface somewhere where on a specific main XBMC machine you can enable a mode like "Library Sharing" as something you can enable similar to allowing remote controlling of XBMC and can enable that machine as the "Host" which controls the main library.

Then, you could have another option for the "Client" machines where you can just enter the IP (Or network name) of the Host machine for it to sync with.

Behind the scenes it can just sync up the database between the two.

What I was thinking of trying on my system was actually mounting the folders where the database is stored as an NFS share in linux (since I have Ubuntu machines, and XBMC Live is based off Ubuntu) and then just have other machines mount the same NFS shared folder.

Does anyone know the specifics of this and if it would work? Would I be able to just mount the library folder of my main Host machine on a Client machine? Does anyone know where this library folder would be?

I want to avoid doing all config since different machines would have different video and audio settings etc.

RockDawg · 2011-09-20 14:16

Hyram Wrote:No, you haven't misunderstood. I feel that with most rendering duties being shunted off to the GPU (meaning there is CPU clock-cycles to spare by the metric tonne) and the prevalence of gigabit ethernet in the home LAN, any performance hit incurred by reading metadata directly instead of a database would be negligible.

You feel or you know the performance hit would be negligible? I can't imagine why you would think that reading (potentially) thousands of xml files wouldn't be massively slower than queering a single database. Databases are designed for optimum retrieval of large amounts of data.

Here's a scenario; let's say I'm looking at the movie info for Armageddon and in the cast list I click on Bruce Willis to see all the movies I have that he is in. I currently have ~975 movies and it takes XBMC around 1-2 seconds to return the list of movies starring Bruce Willis from the database. Imagine how long it would take to scan the xml files for all 975 movies which are spread out across 8 hard drives most of which are spun down looking for the 12 or so I may have with Bruce Willis in them. Every time you want to browse movies by genre XBMC would have to read every xml file again just to generate the list of genres. And what about data that isn't stored in the xml file? Do you ever use the resume play feature in XBMC? That data is stored only in the database, not in the XML file.

I can't see a scenario where doing away with the database would be a good idea and I bet that it will never happen.

Ray_N · 2011-10-26 02:31

Seeing as though the synchronization feature is a common desire among us here, and the solution on the famous article on Lifehacker seems to not be complete and elude even some of us knowledgeable in SQL, operating systems, XML and networking, I've taken the information on the famous article and created and XBMC wiki page to help us through.
http://wiki.xbmc.org/index.php?title=Syn...iple_XBMCs
The original article was written back on September 2010 and some things have changed since then. Now as an XBMC wiki article we can all contribute and fix any errors that may exist. Let's help fix it.

Enjoy,
r

Paradoxgrowth · 2011-11-10 17:35

Ray_N Wrote:Seeing as though the synchronization feature is a common desire among us here, and the solution on the famous article on Lifehacker seems to not be complete and elude even some of us knowledgeable in SQL, operating systems, XML and networking, I've taken the information on the famous article and created and XBMC wiki page to help us through.
http://wiki.xbmc.org/index.php?title=Syn...iple_XBMCs
The original article was written back on September 2010 and some things have changed since then. Now as an XBMC wiki article we can all contribute and fix any errors that may exist. Let's help fix it.

Enjoy,
r

Sorry but SQL does not do the entire job. Sad

SQL only syncs the library information. I do not use the library because all of my downloads have incorrect tag info. I always browse the file structure. I have so much coming in and out, I would need to hire a full time team to fix my tags.

So i am screwed, anything outside of the library (100% of what i watch) goes un-synced. WHY IS THIS NOT A FEATURE YET!!! HELLO EVEN LIFE HACK IN THE 80'S WROTE A WALK THROUGH Angry

**Ned Scott** · 2011-11-11 01:21

Paradoxgrowth Wrote:Sorry but SQL does not do the entire job. SQL only syncs the library information. I do not use the library because all of my downloads have incorrect tag info. I always browse the file structure. I have so much coming in and out, I would need to hire a full time team to fix my tags.

So i am screwed, anything outside of the library (100% of what i watch) goes un-synced. WHY IS THIS NOT A FEATURE YET!!! HELLO EVEN LIFE HACK IN THE 80'S WROTE A WALK THROUGH

HELLO WHAT THE HECK AER YOU TALKING ABOUT MAN ZOMG TACOS BACON

It will still sync your watched ticks in file mode. What else are you trying to sync? Keep your media in one place on your network.

Also, check out a media manager such as Ember to fix your "tags" (do you mean file name?) and stuff.

darkscout · 2011-11-11 03:52

Well, you better not upgrade to Eden because file mode is gone.
I also don't know how you get everything screwed up. Sickbeard and Couchpotato keep everything straight and XBMC has only scraped 1-2 things wrong for me ever. One was the movie Red (because I have both versions) and the other is usually a UK/US version of a TV show.

Also, someone take a look at my bug report:
http://trac.xbmc.org/ticket/12202

Hot off the github compiled an hour ago.

hippojay Fan Posts: 301 Joined: Mar 2008 Reputation: 13 Location: Sheffield, UK	2011-09-10 00:13 Post: #41 Hyram Wrote:Is there an option to force XBMC to not use its internal library, and keep all of its metadata alongside the media instead? If not, there should be. It'd simplify things no end! It would slow down functionality though, as without an internal library you'd need to cycle and read through those files. Unless I've misunderstood your idea?
	find quote

Hyram Junior Member Posts: 45 Joined: Jun 2011 Reputation: 0	2011-09-10 06:19 Post: #42 No, you haven't misunderstood. I feel that with most rendering duties being shunted off to the GPU (meaning there is CPU clock-cycles to spare by the metric tonne) and the prevalence of gigabit ethernet in the home LAN, any performance hit incurred by reading metadata directly instead of a database would be negligible.
	find quote