Pathsubstitution for SMB thumbnails and forcing a full library refresh?

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
AbRASiON Offline
Fan
Posts: 314
Joined: Jan 2004
Reputation: 0
Post: #1
I finally configured XBMC last night to have pathsubstitution configured even for the local copy, so my advancedsettings.xml file could be identical on all 3 computers.


<pathsubstitution>
<substitute>
<from>special://masterprofile/Thumbnails/</from>
<to>SMB://scott:PASSWORD@192.168.0.16/C$/Users/scott/AppData/Roaming/XBMC/userdata/thumbnails/</to>
</substitute>
</pathsubstitution>


My thumbnails are 'broken' because it seems that XBMC (and more importantly Windows) had issues with such a huge amount of connections to an SMB share. I updated my full library overnight and all the data scraped ok but the thumbnails did not all update. Data I know should have thumbnails is simply missing it (example Dexter, in a proper renamed format)
This is applicable to both the local machine holding the library and the other machines connecting to it.
Even when I remove the pathsubstitition and revert back to accessing the same path but locally (ie: normal defaults for xbmc) it's clearly showing many thumbnails being missing.

So I'm now stuck with a perfectly setup database in MySQL but thumbnails which need a full refresh.


I have 3 questions.
1. is there any way around this SMB issue I've discovered? Am I doing something wrong? - It's working just not 'properly' and I'm sure it's a concurrent connection or SMB limitation issue, the SMB server simply can't let a program talk to it and suddenly request 150 thumbnails from a driectory quickly and cleanly. XBMC isn't throttling itself and it's also not "re-asking" for the files when it was denied by SMB (this again, is my theory on why the pathsubstitution seems to be working but not 'fully' - I could be entirely wrong)


2. assuming I can't fix it, can I just let my local PC (server) holding the mysql database, do a full complete refresh, automatically updating the thumbnails to local disk rather than the exact same spot but via SMB. Then, once this is done, if I copy that thumbnail directory to another machine, will it work (is the naming convention for thumbnails identical always? Will the other PC be able to work with a library on another PC but local thumbnails.?)

and finally
3. how the hell do I do a FULL refresh of my database without wiping it? I've tried cleaning it and I've tried scanning for new content and updating. It seems to double check that the records exist in MySQL but it refuses to acknowledge or recognise that a large portion of the thumbnails can't be found and won't go and 'grab them again' <- this is what's killing me the most! I could test out question #2 in a cinch if I could get it to damn well pull all my thumbnails down again.


Does all this make sense?
find quote
AbRASiON Offline
Fan
Posts: 314
Joined: Jan 2004
Reputation: 0
Post: #2
I have manually fiddled around and confirmed the naming convention for files appears to be the same. I am unsure how it generates the names and if it will bite me in the ass later though.
Fanart for Dexter appears to be "0b39a43a.tbn" Californication "e8cb9b8b.tbn"

Not to be a whiner but considering no responses yet, I am resigned to wipe my entire SQL database (again) re-set up the application entirely from scratch and use pathsubstitiution for every single option I can EXCEPT thumbnails and see if this works (Then I only have to manually copy the thumbnails over from the server to my workstation once a week or so)

I'd love to have solved question 1 and identify how or why this all screwed up, but I suspect my SMB limitation theory being fairly sound.
I will report back in this thread if I manage to solve this issue and hopefully help others.
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,228
Joined: Oct 2003
Reputation: 177
Post: #3
If you have invalid thumbs (i.e. you have thumbnail such as the ones you mention for Dexter, but they're corrupt or some such) then your best bet is to start over, yes.

If you have missing thumbs then one option is to export your data to separate files (this will generate .nfo files and copy the thumbs/fanart that you do have next to your movie files). Then drop your db and rescan. Note the disadvantage of doing this: The .nfo files may come back to bite you later in the case that you want to get updated information or rescan afresh from the web enmass.

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
AbRASiON Offline
Fan
Posts: 314
Joined: Jan 2004
Reputation: 0
Post: #4
I've actually just learnt a hard lesson in ticking the option "Movies are in separate folders that match the movie title"

Not being entirely sure what it did, (my data is a real mess) it managed to make some things recognised quite incorrectly.
I tried several solutions to fix this but unfortunately nothing worked, best thing I've found is to set the source folder to data type "none" then "disable from scanning" - it then fully cleans the database, deletes data and I can set it properly, leaving the "seperate folders" option definitely unticked.

I'm nearly there,..... painfully so though, I hope my posts help others on this.
find quote
thrak76 Offline
Moderator
Posts: 1,374
Joined: Mar 2011
Reputation: 25
Location: Lake Michigan Shoreline
Post: #5
(2012-03-30 02:05)AbRASiON Wrote:  [.....]
3. how the hell do I do a FULL refresh of my database without wiping it? I've tried cleaning it and I've tried scanning for new content and updating. It seems to double check that the records exist in MySQL but it refuses to acknowledge or recognise that a large portion of the thumbnails can't be found and won't go and 'grab them again' <- this is what's killing me the most! I could test out question #2 in a cinch if I could get it to damn well pull all my thumbnails down again.


Does all this make sense?

Before setting up my MySQL database I had a similar problem with my thumbnails. The only solution i found to get all my thumbnails back was to delete the textures.db file from the userdata folder. Immediately after the next restart of xbmc, i had all my thumbnails back. Just a suggestion. I'm not sure if it works the same with all your pathsubs and such... might be worth a try though.
(This post was last modified: 2012-03-30 06:46 by thrak76.)
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,228
Joined: Oct 2003
Reputation: 177
Post: #6
That is completely unrelated. Video thumbs or fanart don't touch the texturedb (until my changes are in, ofcourse) - it would only affect local skin-hack thumbs.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
AbRASiON Offline
Fan
Posts: 314
Joined: Jan 2004
Reputation: 0
Post: #7
I've now moved to thumbs not being sync'd - I just don't think Samba can do with that many sudden and concurrent hits thrashing it.
Things have been significantly more reliable since doing this.

I just manually copy the directory from one PC to the other every couple of days.
find quote
smilerz Offline
Junior Member
Posts: 49
Joined: Mar 2010
Reputation: 0
Post: #8
I did write a script that will refresh the NFO data without trashing the rest of the database.

http://smilerz91.blogspot.com/2012/04/md...brary.html

On a side note - once I moved to shared thumbnails I had to delete all of them and let XBMC recreate them.
find quote
PatrickBateman Offline
Posting Freak
Posts: 1,677
Joined: Mar 2011
Reputation: 17
Location: Another Earth
Post: #9
(2012-04-16 02:09)smilerz Wrote:  I did write a script that will refresh the NFO data without trashing the rest of the database.

http://smilerz91.blogspot.com/2012/04/md...brary.html

On a side note - once I moved to shared thumbnails I had to delete all of them and let XBMC recreate them.

Is there any way this script could be turned into an xbmc plugin / script that can be ran from the plugins section?

This would solve many peoples problems... My self included Smile

Fingers crossed.
find quote
smilerz Offline
Junior Member
Posts: 49
Joined: Mar 2010
Reputation: 0
Post: #10
I'm sure someone could do it - unfortunately, I can barely script in PowerShell. Python is completely beyond my skill.
find quote
AbRASiON Offline
Fan
Posts: 314
Joined: Jan 2004
Reputation: 0
Post: #11
I set up Microsoft sync toy tools thingo to run a daily re-sync of directories, so it's mostly solved it for me.
find quote