XBMC Community Forum
Directory Watchdog - Background file monitor service to initiate auto update library? - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Feature Suggestions (/forumdisplay.php?fid=9)
+--- Thread: Directory Watchdog - Background file monitor service to initiate auto update library? (/showthread.php?tid=66348)

Pages: 1 2 3 4 5 6 7 8 9 10 11

- althekiller - 2010-01-09 23:33

Watching inodes is a non-trivial task. It's not portable across platforms and there are implications when the file system is network based, which a majority of users have. Simply stat()ing the directories in an event loop can be expensive too. Again, especially for network sources.

The best solution I've thought of is to have an event client that runs on the system where your media actually resides. It does all of the dir watching, and sends XBMC a request to scan the newly added directory, translating paths if necessary. So say you have a new dir /media/movies/NewMovie where /media/movies is shared on SERVER over SMB as Movies. When the file transfer to this directory completes (also non-trivial to detect) the EC fires of an execute builtin request for XBMC to scan smb://SERVER/Movies/NewMovie. Problems of course arise here when you have multiple movies being transfered to the watch dir so you may get a scan request while one is in progress. Currently there's no queuing in the scanner, so (I think) the first scan will be cancelled. There's also the fact that the event server is UDP based, so your client has no idea whether the request was actually received. You could alleviate this by using the HTTP-API and caching the failed scan requests client-side.

- Hitcher - 2010-01-10 02:02

What about a timed update set by the user?

Daily, hourly, etc.

- idlehands - 2010-01-10 03:42

there are some torrent clients that support twitter. could be horribly convoluted but tweeting when a torrent is done, and just having xbmc monitor that feed could be interesting.

- althekiller - 2010-01-10 03:56

You can do time based stuff the same way. httpapi via cron on linux. I'm sure there are ways on those other OSes as well.

- robweber - 2010-01-10 04:11

I've started this patch that will fire off a library update based on a user set timer. Once the update runs the timer begins again. It uses the current AlarmClock functionality of XBMC, I haven't had any feedback on the trac about it though. Any thoughts are welcome.

http://trac.xbmc.org/ticket/8431

- franciscohs - 2010-01-10 20:40

althekiller Wrote:Watching inodes is a non-trivial task. It's not portable across platforms and there are implications when the file system is network based, which a majority of users have. Simply stat()ing the directories in an event loop can be expensive too. Again, especially for network sources.

The best solution I've thought of is to have an event client that runs on the system where your media actually resides. It does all of the dir watching, and sends XBMC a request to scan the newly added directory, translating paths if necessary. So say you have a new dir /media/movies/NewMovie where /media/movies is shared on SERVER over SMB as Movies. When the file transfer to this directory completes (also non-trivial to detect) the EC fires of an execute builtin request for XBMC to scan smb://SERVER/Movies/NewMovie. Problems of course arise here when you have multiple movies being transfered to the watch dir so you may get a scan request while one is in progress. Currently there's no queuing in the scanner, so (I think) the first scan will be cancelled. There's also the fact that the event server is UDP based, so your client has no idea whether the request was actually received. You could alleviate this by using the HTTP-API and caching the failed scan requests client-side.

I had no idea it was such a complex task and I see you've already though about it extensively. Thanks for the explanation.

- joshua.lyon - 2010-01-11 21:54

I made a blog post that explains step-by-step how to setup automatic library updates for XBMC Live. Check out the Automatic Library Updates for XBMC Live post on my site and if you have any questions, either post them in the comments or on this thread. I should note that it is updating the library on a timed basis and is not watching particular folders for changes.

I agree that this should be part of the core XBMC distribution, but for now I can live with the simple 'script' I'm using in my how to. Wink It was easy to setup and now handles everything on it's own.

PS. If you don't know how to use SSH to access your box, you can use the first part of my post about Installing SABnzbd+ on XBMC Live which describes how to access XBMC via SSH (or directly from an attached keyboard).

- CrashX - 2010-01-11 22:14

robweber Wrote:Here is a link to the TRAC where I posted a patch for what I've come up with so far. My patch allows you to:
a) enable auto update and define a number of hours 1 - 23.
b) starts a countdown that will update the database the set number hours, and then restart the countdown until disabled

One little problem is that if you restart XBMC the process will not restart itself. This should be a simple enough matter and can be inserted into the startup sequence wherever the check for updating the database on startup is. If this feature looks like something people are actually interested in I'll take the time to do that as well.

http://trac.xbmc.org/ticket/8431

One side note - I do agree that a 'watchdog' feature that would monitor the source and update is the most ideal, but that cross platform this is unlikely. I think the ability to update the library automatically without a restart of the whole system is very close to perfection for most people.


Another way to do using built in xbmc idle timers .. http://trac.xbmc.org/ticket/5969 ..

Idle Do Action
-> Shutdown ( Executes the default shutdown action )
-> Update Library ( already feature request for updating on interval )
-> idle.py


Lost Idle Do Action
-> wakeup.py

- robweber - 2010-01-11 23:18

CrashX Wrote:Another way to do using built in xbmc idle timers .. http://trac.xbmc.org/ticket/5969 ..

Idle Do Action
-> Shutdown ( Executes the default shutdown action )
-> Update Library ( already feature request for updating on interval )
-> idle.py


Lost Idle Do Action
-> wakeup.py

I read over the discussion on the ticket you posted the link to. If nothing else I like the idea that the update would not happen unless the system is idle. this would help with some of the conerns althekiller raised with my patch (ie if you are using XBMC and an update happens every hour or so it may interrupt things you want it to be doing).
I like the idea that I could set the library to update every 5 - 8 hours and it would happen unless I was using the system for something else. I guess in my head this type of solution works really well for people who add content and then want it to just 'show up' for friends and family members who may also use the system.

Directory Watchdog - Service that monitor changes of file sources in the background? - CrashX - 2010-01-15 18:21

Here is some sample code to monitor files/directory in C++ for Windows ?

http://www.codeguru.com/forum/showthread.php?t=436716

Anyone know how to do this linux ?

With the above code, we can enhance xbmc so that it will automatically update library when a file/directory change occurs.

I was thinking adding this code to XBMC when we add sources.