XBMC Library Updater Addon

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
robweber Online
Posting Freak
Posts: 822
Joined: Sep 2009
Reputation: 26
Post: #286
Yes, as long as all the clients connect to your shared library they will be updated. I currently do this at my house with 1 main xbmc box that runs 24/7 and several clients around the house that I just turn on as needed. The image caching system takes care of the images so each machine will update it's image cache as needed with the new content.
find quote
tknorris Offline
Member
Posts: 63
Joined: May 2012
Reputation: 1
Post: #287
I think I found a long-standing rounding error in this addon. I've been using it forever (and it's great btw) but for the longest time I've gotten the occasional duplicate library update that I couldn't explain. Tonight I finally decided to look into it and I think the problem comes in with this line (line 104 of service.py):
Code:
self.last_run = time.time() - (time.time() % 60)

That code is intended to capture the minute of the last run, but sometimes the float expression result will be just before the actual minute and thus croniter.get_next returns the next run as the one that just completed rather than the one that should next be scheduled. This then triggers the addon to update library again on the next minute click.

you can see some examples of what I'm talking about in this code snippet:
Code:
>>> print time.time();time.time() - (time.time() % 60);ceil(time.time() - (time.time() % 60));(time.time()+1) - (time.time() %60)
1394598973.24
1394598959.999999
1394598960.0
1394598960.9999962

This also shows the potential fixes. You can either add 1 to time.time() (if you don't want to import math) or you can take the ceiling of the existing result to force it to round up

Also, here's a log snippet that shows the problem occurring (in my case my cron expression is */15 * * * *). Note, I added some logging to see when last_run was read and written.
Code:
00:15:00 T:139895397181184  NOTICE: service.libraryautoupdate: Starting network check
00:15:00 T:139895397181184  NOTICE: service.libraryautoupdate: Update Video Library
00:15:00 T:139895397181184  NOTICE: Next Run: 1394598600.0
00:15:00 T:139895380395776  NOTICE: Thread CVideoInfoScanner start, auto delete: false
00:15:00 T:139895380395776  NOTICE: VideoInfoScanner: Starting scan ..
00:15:01 T:139895380395776  NOTICE: VideoInfoScanner: Finished scan. Scanning for video info took 00:00
00:15:01 T:139895397181184  NOTICE: Writing Last Run: 1394597699.999999
00:15:02 T:139895397181184  NOTICE: service.libraryautoupdate: update timers
00:15:02 T:139895397181184  NOTICE: Next Run: 1394597700.0
00:15:09 T:139894118627072  NOTICE: AddOnLog: MythTV cmyth PVR Client: Process - Event unknown, databuf:
00:16:00 T:139895397181184  NOTICE: Previous line repeats 5 times.
00:16:00 T:139895397181184  NOTICE: service.libraryautoupdate: Starting network check
00:16:00 T:139895397181184  NOTICE: service.libraryautoupdate: Update Video Library
00:16:00 T:139895397181184  NOTICE: Next Run: 1394598600.0
00:16:00 T:139895388788480  NOTICE: Thread CVideoInfoScanner start, auto delete: false
00:16:00 T:139895388788480  NOTICE: VideoInfoScanner: Starting scan ..
00:16:00 T:139895388788480  NOTICE: VideoInfoScanner: Finished scan. Scanning for video info took 00:00
00:16:00 T:139895388788480  NOTICE: Thread Jobworker start, auto delete: true
00:16:01 T:139895397181184  NOTICE: Previous line repeats 1 times.
00:16:01 T:139895397181184  NOTICE: Writing Last Run: 1394597760.000000
00:16:02 T:139895397181184  NOTICE: service.libraryautoupdate: update timers
00:16:02 T:139895397181184  NOTICE: Next Run: 1394598600.0

Hope this helps fix this issue. I'm just glad at this point I finally understand why it happens as it's driven me crazy for the longest time.
(This post was last modified: 2014-03-12 08:11 by tknorris.)
find quote
robweber Online
Posting Freak
Posts: 822
Joined: Sep 2009
Reputation: 26
Post: #288
That is great, thanks for taking the time to actually dig in to why this is happening. I'll be honest I've never noticed this myself but I'm probably just not being very observant. The line of code you have identified has been in the addon since practically the beginning so I'm sure this is a long-standing bug as you've indicated.

Out of the two ways you've specified to fix it, which do you think is the best? Adding 1 seems the easiest without doing a lot of extra computations. I'll try to update this and get a fix out for Frodo and Gotham this week. I'll make sure you get the credit in the changelog too!
find quote
tknorris Offline
Member
Posts: 63
Joined: May 2012
Reputation: 1
Post: #289
After thinking about this last night, I think the reason this is happening in the 2nd call to time.time() sometimes returns a slightly larger result than the first call since a tiny bit of time has clicked away. If that theory is correct then neither the +1 nor the ceiling is really required and this code change would fix it:

Code:
now = time.time()
self.last_run = now - (now % 60)

I've been running this change for the last hour on my HTPC and haven't gotten any duplicate runs. So that's probably the best fix.
find quote
robweber Online
Posting Freak
Posts: 822
Joined: Sep 2009
Reputation: 26
Post: #290
That makes some sense. Even though those calls are very close together there would be some tiny fraction of time that would have gone by. If you hit the marks just right you'd get the wrong minute. I'll implement your proposed change and run it on my system fora while too. I'll bump up my scan rate to every 15 minutes for a few hours and keep an eye on it.
find quote
GTHvidsten Offline
Junior Member
Posts: 4
Joined: Mar 2014
Reputation: 0
Post: #291
In Gotham Beta 2 it seems that cleaning libraries doesn't quite work. The add-on updates the library with new content, but I also have content that is not removed when the files are removed.
find quote
robweber Online
Posting Freak
Posts: 822
Joined: Sep 2009
Reputation: 26
Post: #292
Can you confirm that the clean command is even being issued? Library cleaning from Frodo to Gotham changed in the XBMC core, could be a bug in Gotham. Can you do a manual clean under Settings->Video and see if your items still remain? If this happens then it's a problem with the cleaning function of xbmc.
find quote
litfoo Offline
Senior Member
Posts: 153
Joined: Dec 2013
Reputation: 0
Post: #293
Thanks for this excellent add-on - great to have further control of the library

not sure if I am completely missing the point but I am not sure how I can achieve my desired outcome.

My XBMC on Windows is scheduled to restart each morning at 06:15

I just want to implement a delay of 2 mins each day to update the video library

It seemed to do this on the first time but not once XBMC restarted the next day - just says update in xx.

I do not want to say update in 24 hours as sometimes I have to restart XBMC intra day

I just want to set a specific time to update the video library or update on xx delay after XBMC restart

Intel NUC DC3217IYE, 8GB Corsair RAM, Crucial 120GB mSATA, Windows 8.1 - XBMC Frodo 12.3- Aeon MQ 5 / Aeon Nox, JVC HD-550 Projector, Beamax M Series 10015 Screen, Synology DS1812+, Qnap TS-410, Logitech Harmony 1000 & RF Extender, Flirc+ USB dongle

[Image: all-thin-fanart.jpg]
(This post was last modified: 2014-04-22 17:21 by litfoo.)
find quote
zer04c Offline
Junior Member
Posts: 24
Joined: Jan 2013
Reputation: 0
Post: #294
Hi, thanks for all the work on this great addon.

I'm using a raspberry pi and library updating really grinds the things to a halt. Is there anyway to get the addon to run when I'm not using the UI ? Either in during me watching something or when the device is idle ?
Many thanks
find quote
robweber Online
Posting Freak
Posts: 822
Joined: Sep 2009
Reputation: 26
Post: #295
(2014-04-22 17:15)litfoo Wrote:  Thanks for this excellent add-on - great to have further control of the library

not sure if I am completely missing the point but I am not sure how I can achieve my desired outcome.

My XBMC on Windows is scheduled to restart each morning at 06:15

I just want to implement a delay of 2 mins each day to update the video library

It seemed to do this on the first time but not once XBMC restarted the next day - just says update in xx.

I do not want to say update in 24 hours as sometimes I have to restart XBMC intra day

I just want to set a specific time to update the video library or update on xx delay after XBMC restart

There best way to do what you are asking is to use the advanced timer and set a cron-type expression to run a few minutes after 6:15 each night. This would look something like 20 6 * * *

Is there any reason you need a 2 minute delay after xbmc boots? If your network is active you could simply check the "update on startup" box under Settings->Video that is built in to xbmc. This would run the library scan process each time xbmc boots up. I'm not implementing a "start at boot" option into this addon as xbmc already has this in the core.

(Yesterday 12:59)zer04c Wrote:  Hi, thanks for all the work on this great addon.

I'm using a raspberry pi and library updating really grinds the things to a halt. Is there anyway to get the addon to run when I'm not using the UI ? Either in during me watching something or when the device is idle ?
Many thanks

There is a checkbox to let the addon run during video watching but nothing that would make it run specifically then. In all honesty most people dont want it to run when they are watching something because it could lag performance of the video due to the processing power needed as well. The RPI is kind of a underpowered machine, and depending on the skin and other services you have running it could just be taxing the little processor too much.
find quote
weardark Offline
Junior Member
Posts: 3
Joined: Nov 2013
Reputation: 0
Post: #296
Thanks for this great add-on. I love it.

I recently noticed that 1 of my 2 video sources has stopped updating automatically.

My video library consists of 2 sources - 'Series' and 'Movies'. Both of these sources are mapped network drives. My Series source seems to catch new content reliably but my Movies source has completely stopped updating.

When I add a new movie to my collection I have to go to Videos > Files > Movies.. find the file (which is always listed with my other library titles so I know XBMC sees it) and right click on it to start the scraper. This is a minor nuisance, but my recently added list doesn't update when I manually add the titles either.

Can anyone point me in the right direction to debug this issue?

Thanks!
find quote
litfoo Offline
Senior Member
Posts: 153
Joined: Dec 2013
Reputation: 0
Post: #297
(Yesterday 15:17)robweber Wrote:  There best way to do what you are asking is to use the advanced timer and set a cron-type expression to run a few minutes after 6:15 each night. This would look something like 20 6 * * *

Is there any reason you need a 2 minute delay after xbmc boots? If your network is active you could simply check the "update on startup" box under Settings->Video that is built in to xbmc. This would run the library scan process each time xbmc boots up. I'm not implementing a "start at boot" option into this addon as xbmc already has this in the core.

Thanks for this - the 20 6 * * * makes sense. (6.20am each day)

I did have 'update on startup' but due to what looked like some NFS issues on Windows/XBMC boot putting in a delay on the library update seems to the workaround/solution Thanks for the suggestion though

Intel NUC DC3217IYE, 8GB Corsair RAM, Crucial 120GB mSATA, Windows 8.1 - XBMC Frodo 12.3- Aeon MQ 5 / Aeon Nox, JVC HD-550 Projector, Beamax M Series 10015 Screen, Synology DS1812+, Qnap TS-410, Logitech Harmony 1000 & RF Extender, Flirc+ USB dongle

[Image: all-thin-fanart.jpg]
find quote
robweber Online
Posting Freak
Posts: 822
Joined: Sep 2009
Reputation: 26
Post: #298
(Yesterday 15:33)weardark Wrote:  Thanks for this great add-on. I love it.

I recently noticed that 1 of my 2 video sources has stopped updating automatically.

My video library consists of 2 sources - 'Series' and 'Movies'. Both of these sources are mapped network drives. My Series source seems to catch new content reliably but my Movies source has completely stopped updating.

When I add a new movie to my collection I have to go to Videos > Files > Movies.. find the file (which is always listed with my other library titles so I know XBMC sees it) and right click on it to start the scraper. This is a minor nuisance, but my recently added list doesn't update when I manually add the titles either.

Can anyone point me in the right direction to debug this issue?

Thanks!

Best thing to do is get a debug log of the video scanner running and post it on xbmclogs.com. Making a post in the support forum for your operating system may attract the attention of a dev, this is something to do with the scanner in the xbmc core.
find quote
zer04c Offline
Junior Member
Posts: 24
Joined: Jan 2013
Reputation: 0
Post: #299
(Yesterday 15:17)robweber Wrote:  There is a checkbox to let the addon run during video watching but nothing that would make it run specifically then. In all honesty most people dont want it to run when they are watching something because it could lag performance of the video due to the processing power needed as well. The RPI is kind of a underpowered machine, and depending on the skin and other services you have running it could just be taxing the little processor too much.

Thanks , ironically during playback is the time I notice it least. I think it because the RPI is usually running between 40-90 % when using the UI but only about 25% during playback due to the decoding on the hardware.
Would there be a way to activate when the device is idle instead. ?
find quote
Post Reply