2 XBMC PCs and 1 Database on the shared NAS

  Thread Rating:
  • 7 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
bradvido88 Offline
Donor
Posts: 794
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #31
charrua Wrote:Good! With each new automated scan you are adding only the media that it's not in the library central db already, so be aware that for thumbs and fanart to be in each local cache you'll still have to perform the steps listed before.
Thanks for clarifying this. I didn't realize automated scans do not also UPDATE media in the library. Oh well. Maybe I tried some sort of folder redirection or syncing for the thumbnails/fanart.

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
find quote
MrDVD Offline
Senior Member
Posts: 266
Joined: Apr 2004
Reputation: 0
Post: #32
Hallo, is it maybe possible to store the fanarts, posters and so on also central and link this folder to each clients default place ? (like /home/username/.xbmc/userdate/Thumbnails/Video for linux ?)
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #33
MrDVD Wrote:Hallo, is it maybe possible to store the fanarts, posters and so on also central and link this folder to each clients default place ? (like /home/username/.xbmc/userdate/Thumbnails/Video for linux ?)

That could be an option, the problems I see in this solution so far are:
1-The reason for an image cache in the first place is speeding up loading times, and I'm not sure if that would be the case with a networked repository instead of a local one.
2-If for some reason you loose the connection to the networked cache, XBMC will end up not showing any pictures.
find quote
MrDVD Offline
Senior Member
Posts: 266
Joined: Apr 2004
Reputation: 0
Post: #34
charrua Wrote:That could be an option, the problems I see in this solution so far are:
1-The reason for an image cache in the first place is speeding up loading times, and I'm not sure if that would be the case with a networked repository instead of a local one.
2-If for some reason you loose the connection to the networked cache, XBMC will end up not showing any pictures.

to 1 - is an GB network realy that much slower than in most cases used 2,5" hdd to have an silent htpc ? Sure thats dependent on the number of clients the are connected at the same time.

to 2 - but when i loose the connection to the network in my case, XBMC will end up showing nothing, In fact it will not see any show, movie too Smile
So in my case, the missing thumbnails are the smallest problem. Smile
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #35
MrDVD Wrote:to 1 - is an GB network realy that much slower than in most cases used 2,5" hdd to have an silent htpc ? Sure thats dependent on the number of clients the are connected at the same time.
I think the most important factor to consider regarding loading speed of images is not really bandwidth, but latency (access time) and there is a difference between disk and network access times, specially noticeable in small sized files like images, a 1 sec. access time to a movie between the time you click the poster and the time XBMC starts playing it seems ok, but the same 1 sec.access for each image loaded(consider the big number of images each screen loads, specially with some skins) would most probably slow down the GUI a lot.
MrDVD Wrote:to 2 - but when i loose the connection to the network in my case, XBMC will end up showing nothing, In fact it will not see any show, movie too Smile
So in my case, the missing thumbnails are the smallest problem. Smile
That is true, but one thing is not being able to watch the movies/shows and much worse than that is not being able to even browse through your media library properly.
If your electricity is gone wouldn't be nice at least to have your phone line working? Smile

That being said, IMHO yours could be a possible and temporary solution to the thumbnail problem until the new image cache system is ready to go.
Testing will tell...
(This post was last modified: 2010-03-24 20:03 by charrua.)
find quote
bradvido88 Offline
Donor
Posts: 794
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #36
charrua Wrote:1-The reason for an image cache in the first place is speeding up loading times, and I'm not sure if that would be the case with a networked repository instead of a local one.

I agree that the image cache has to stay on the local hard drive for latency issues.

Here's what I'm proposing, let me know your thoughts:
When an XBMC instance finds a new movie and it's not in the database, it should then scrape all the movie information and update the database. For the thumbs/fanart/images, they should be stored locally in its cache, but also written to the database (as BLOBS). If any of these images are ever updated, they should be updated in the local image cache and in the database (along with a last-modified field).

Now when a second client comes along, we don't have to worry about client-specific URI's; it only needs to know how to connect to the database. It sees the new movie in the database, and checks to see if it has any thumbs/fanart etc. stored in its local cache.
From here, it has 3 options:
  1. If it doesn't have them stored in its cache, then it downloads them from the database and writes them to its cache.
  2. If it has them in its cache, but the file has been modified (based on a last-updated field), then it updates it's cached image.
  3. If it already has them and they're all up to date, then it simply skips.


This way the local, speedy cache gets preserved on all XBMC instances. It also keeps all images in sync across all XBMC instances. When an image needs to be downloaded for any subsequent clients, it gets downloaded quickly from the local MySQL server, as opposed to across the internet. The client then has a local cached copy for even faster subsequent reads.

Thoughts?

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
(This post was last modified: 2010-03-25 15:21 by bradvido88.)
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #37
bradvido88 Wrote:I agree that the image cache has to stay on the local hard drive for latency issues.

Here's what I'm proposing, let me know your thoughts:
When an XBMC instance finds a new movie and it's not in the database, it should then scrape all the movie information and update the database. For the thumbs/fanart/images, they should be stored locally in its cache, but also written to the database (as BLOBS). If any of these images are ever updated, they should be updated in the local image cache and in the database (along with a last-modified field).

IMHO it's a very good idea. If I remember correctly firnsy already proposed a similar approach some time ago (but he wasn't so clear about it, and then the lightning took his words away Smile). Only two possible cons that I can think of:
-The size of the db will grow considerably, but that's not so bad.
-I'm not sure about MySQL performance when storing/querying BLOB fields.
find quote
bradvido88 Offline
Donor
Posts: 794
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #38
charrua Wrote:IMHO it's a very good idea. If I remember correctly firnsy already proposed a similar approach some time ago (but he wasn't so clear about it, and then the lightning took his words away Smile). Only two possible cons that I can think of:
-The size of the db will grow considerably, but that's not so bad.
-I'm not sure about MySQL performance when storing/querying BLOB fields.

Ahh yes, that damn lightning! Shocked Erased a lot of good discussion!

For your two points:
1. The size will grow, but I wouldn't consider it to be a huge growth. So what if you add 100MB to you database.... that's barely any disk space by today's standards.

2. The bottom line is that the performance will still be many times faster than grabbing the images from an internet site over and over again for every xbmc instance.

It just seems to me that storing the actual image instead of the image url is much more streamlined and smooth. At least it looks that way to the to the user.
It would be pretty unintuitive if a user adds a movie on one XBMC instance, then decides to finish it on another instance. But when they get the the second instance, it takes 15 seconds for the XBMC to query TMDB and sync all the same images they already saw on their first instance. With mysql, it will take only a second or two.

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #39
bradvido88 Wrote:Ahh yes, that damn lightning! Shocked Erased a lot of good discussion!
For your two points:
1. The size will grow, but I wouldn't consider it to be a huge growth. So what if you add 100MB to you database.... that's barely any disk space by today's standards.
Well, in my case the thumbnails folder size as it is now is more than 3Gb.
Probably the new image cache system being developed by jmarshall would be intelligent enough to purge unused images from the cache, reducing that size a little bit.

bradvido88 Wrote:2. The bottom line is that the performance will still be many times faster than grabbing the images from an internet site over and over again for every xbmc instance.
That's true, but you should compare the performance with the alternative of having a central cache file based instead of stored in the db.

bradvido88 Wrote:It just seems to me that storing the actual image instead of the image url is much more streamlined and smooth. At least it looks that way to the to the user.
It would be pretty unintuitive if a user adds a movie on one XBMC instance, then decides to finish it on another instance. But when they get the the second instance, it takes 15 seconds for the XBMC to query TMDB and sync all the same images they already saw on their first instance. With mysql, it will take only a second or two.
Yes, I agree. But I also like the idea to store the URIs in the db. Let's see what the future brings... count me in for testing Smile
find quote
bradvido88 Offline
Donor
Posts: 794
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #40
charrua Wrote:Well, in my case the thumbnails folder size as it is now is more than 3Gb.
Holy ****!
That's one big library you must have to generate that much thumbnail data Smile

I'm excited for the future too! I wish i could actually contribute patches, but for right now my time is limiting me to testing & reporting...

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #41
bradvido88 Wrote:Holy ****!
That's one big library you must have to generate that much thumbnail data Smile
Well, big enough... ~2000 movies, [SIZE=1]~3500 tvshow episodes, [SIZE=1]~2200 musicvideos and [SIZE=1]~3700 music albums in the library + a hole bunch of media files that I don't add to the library but still generate entries in the thumbnail cache everytime I browse them in fileview mode... and consider that I purge the thumbnail cache folder now and then otherwise the cache folder size would be much bigger. Smile
[/SIZE][/SIZE][/SIZE]
(This post was last modified: 2010-03-26 00:51 by charrua.)
find quote
RenZ0R Offline
Senior Member
Posts: 179
Joined: Jan 2008
Reputation: 2
Location: UK
Post: #42
cant you install xbmc onto a server, and run it from there using different profiles for the user that is logged in? ie pc 1, pc 2, etc?
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #43
RenZ0R Wrote:cant you install xbmc onto a server, and run it from there using different profiles for the user that is logged in? ie pc 1, pc 2, etc?
It may be possible, but for sure it'll be super-slow, because it'd have to load everything from the server (GUI images, all the images in the cache,all dlls nedeed by XBMC, etc.) and also is not a solution at all if you intend to run XBMC on clients with different OSs...
find quote
MrDVD Offline
Senior Member
Posts: 266
Joined: Apr 2004
Reputation: 0
Post: #44
Hallo, i run it now for 24h and have to say it works great, when i watch an movie on xbmc1 all other xbmc see it also as watched. For me the serverside image cache works also without any speed problems.

My cache is also ~ 3GB (2000 movies and 10000 episodes == ~ 28TB on data)

If you want an local image cache you can also use rsync under linux to be sure that all caches have the same files in it.
find quote
charrua Offline
Senior Member
Posts: 273
Joined: Nov 2008
Reputation: 14
Post: #45
MrDVD Wrote:Hallo, i run it now for 24h and have to say it works great, when i watch an movie on xbmc1 all other xbmc see it also as watched. For me the serverside image cache works also without any speed problems.

My cache is also ~ 3GB (2000 movies and 10000 episodes == ~ 28TB on data)

If you want an local image cache you can also use rsync under linux to be sure that all caches have the same files in it.
That's amazing, I thought it would be really slow...
Could you give us some more details about your setup? (server specifications, network, installed OSs)
Are all the clients running XBMC from the server simultaneously?
Are the userdata folders of all profiles pointing to different folders and are those folders located also in the same server?
(This post was last modified: 2010-03-25 22:40 by charrua.)
find quote
Post Reply