[RELEASE] Library watchdog

  Thread Rating:
  • 4 Votes - 4.75 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #1
Watch media sources for new files and instantly trigger library updates. Cleaning library when files are deleted is supported as well.
Available from the official add-on repository


Known Issues:

Continuous scanning/cleaning
If this happens you have an application running that continuously reads and/or deletes files in the media source. Remove the offending application or remove the folder from media sources in xbmc. Normally downloading applications and such should not cause this (file modifications are ignored) but certain ones dumps a lot of temporary files to the disk.

Other addons slow to open/wont open at all
Issue in xbmc http://trac.xbmc.org/ticket/14212 Only known to affect Polling

Memory leaking
Issue in xbmc http://forum.xbmc.org/showthread.php?tid=157233 Only affects Polling



FAQ

X doesn't work
1. Enable debug logging
2. Restart XBMC
3. Produce the error
4. Post the complete debug log in this thread and I will try to help.

Implementation details
Window Vista and newer, Linux, OS X, BSD will use event based system calls, Windows XP, others platforms and mounted network shares will use polling.
Source code on github: http://github.com/takoi/xbmc-addon-watchdog

What is the difference between "polling" and "syscall"
Polling involves continuously checking new files, while syscall register a callback. Basically, polling will use more cpu, but usually it shouldn't be noticeable.

If using polling, will it hog the disks and prevent them from spinning down
No. When there's nothing being written to the drives, the OS will usually cache the entire directory structure. If for some reason disk activity persist, you might want to look into increasing the cache space in you OS.

Polling methods:
If you are experiencing performance issues you can select another method for network shares. Here's an explanation of the different polling methods you can choose:

Depth 1: very fast. independent of number of files
Code:
./media source/
    folder a/  <-- detected (will always scan and clean (if enabled). unknown if it was added or removed)
        folder b/  <-- not detected

Depth 2: performance depends on number of directories and files in top folder (media source) only
Code:
./media source/
    folder a/  <-- detected as new
        folder b/  <-- detected (will always scan and clean (if enabled). unknown if it was added or removed)
            folder c/  <-- not detected

Depth inf.: performance depends on total number of directories and files. Will detect everything, also whether a file was added or removed.
(This post was last modified: 2014-04-06 16:12 by takoi.)
find quote
Aenima99x Offline
Donor
Posts: 1,097
Joined: Aug 2007
Reputation: 7
Location: California
Post: #2
Thanks! Seems to be working fine for me. Running Ubuntu 12.04 with a RAID5 for my media and a MySQL DB. All of this is local on the same box.
Question though, you said Movies, I assume it does TV Shows as well since they are both in the same DB? What about Music?

EDIT - Answered my own questions, it added the TV show fine too. Music was not updated though. Any chance of getting that to work?
(This post was last modified: 2012-04-18 05:32 by Aenima99x.)
find quote
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #3
Yeah, it works just with videos for now, but i'll add music support too..
find quote
Aenima99x Offline
Donor
Posts: 1,097
Joined: Aug 2007
Reputation: 7
Location: California
Post: #4
Cool thanks!
find quote
akuiraz Offline
Member
Posts: 76
Joined: Apr 2010
Reputation: 0
Post: #5
any chance that remote shares could be incorporated? my hard drives are on the same computer as my main xbmc install... i just set them up as shares for future xbmc installations in the home with mysql. is there anything i could do to get this working?

[Image: widget]
find quote
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #6
(2012-04-19 00:48)akuiraz Wrote:  any chance that remote shares could be incorporated? my hard drives are on the same computer as my main xbmc install... i just set them up as shares for future xbmc installations in the home with mysql. is there anything i could do to get this working?

Maybe for samba, others probably not. There's some technical difficult with doing this. If someone were to add more functions to xbmc's virtual file system, then it would be an easy task, but this might take long.

The polling method will work on any mounter file system, so one possibility is to just use this instead. However, this will require you to manually mount the shares so it appears local to xbmc. Xbmc does not do this for you, it just opens/close the url as needed. Not sure if this is of any interest then?
find quote
Aenima99x Offline
Donor
Posts: 1,097
Joined: Aug 2007
Reputation: 7
Location: California
Post: #7
Hey Takoi, just wanted to check in and see if you've made any progress on adding the music support? Thanks.
find quote
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #8
(2012-05-09 02:01)Aenima99x Wrote:  Hey Takoi, just wanted to check in and see if you've made any progress on adding the music support? Thanks.

Yeah, I implemented it just a few days ago.:) You'll have to donwload and install it manually from here for now, but I'll push it to official addon repo very soon!
find quote
Aenima99x Offline
Donor
Posts: 1,097
Joined: Aug 2007
Reputation: 7
Location: California
Post: #9
Very cool thanks! Not sure why your addon hasn't gotten more attention, it's really great.
find quote
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #10
ty. There's been 70 downloads, but no one reporting back so I'm gonna just assume everything works perfectly and release it Smile
(This post was last modified: 2012-05-09 17:04 by takoi.)
find quote
Hitcher Offline
Skilled Skinner
Posts: 10,257
Joined: Aug 2007
Reputation: 80
Location: Eastleigh, UK
Post: #11
Can I ask how exactly it know when files are added or deleted?

Thanks.
find quote
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #12
(2012-05-09 17:20)Hitcher Wrote:  Can I ask how exactly it know when files are added or deleted?

Thanks.
Through system calls. inotify on linux, winapi on windows, otherwise it takes a snapshot of a folders 'modified' timestamp every second or so and compare.
find quote
tinybutstrong Offline
Senior Member
Posts: 274
Joined: Sep 2010
Reputation: 0
Post: #13
Cool addon idea, but after installed it, XBMC refuses to close. Stuck on this log message:

18:49:22 T:2232 NOTICE: WATCHDOG: BaseObserver.on_thread_exit()

I wish to have Rotten Tomatoes (user+critics) + IMDB both ratings display in skin.
find quote
takoi Offline
Fan
Posts: 596
Joined: Oct 2009
Reputation: 7
Location: Norway
Post: #14
(2012-05-09 23:51)tinybutstrong Wrote:  Cool addon idea, but after installed it, XBMC refuses to close. Stuck on this log message:

18:49:22 T:2232 NOTICE: WATCHDOG: BaseObserver.on_thread_exit()

Can you please post a complete debug log? I need some more info to see why this happens..
find quote
tinybutstrong Offline
Senior Member
Posts: 274
Joined: Sep 2010
Reputation: 0
Post: #15
Hmm got more info this time, check log bellow:

http://pastebin.com/Cwkqp8QL

I wish to have Rotten Tomatoes (user+critics) + IMDB both ratings display in skin.
find quote
Post Reply