• 1
  • 28
  • 29
  • 30(current)
  • 31
  • 32
  • 197
[RELEASE] Texture Cache Maintenance utility
If you run "nc movies" it will list any movie artwork that needs to be cached. If all artwork is cached and the GUI is not fluent then the lack of fluency is not due to caching, try a lighter skin or faster storage. If you are using a Raspberry Pi, make sure you are using a build with all the latest performance enhancements/speedups.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thank... but "nc" runs the same way then "c" :/
Reply
Big changes, please report any problems (supplying a logfile with verbose option @logfile.verbose=yes really helps!)

Version 1.0.4
  • Add: New Texture JSON API (requires JSON API v6.10.0+).

    Access the Textures*.db database using JSON rather than via SQLite. Use @dbjson=no to force old SQLite behaviour, @dbjson=yes to force JSON API, or leave @dbjson undefined for auto-selection (ie. use JSON API when available, fall back to SQLite when not).

    When using JSON to access Textures*.db, only the following options still require direct file system access to the Userdata\Thumbnails folder: f, r, R, S, X, Xd. All other options (including prune, p and P) now require only a valid xbmc.host and no filesystem access so no need to mount a remote share while pruning a remote client.

    As pruning no longer has (or requires) access to the Thumbnails folder when using JSON API, I've had to remove the "orphan" warning at the end of the prune process.
  • Add: Ability to remove artwork urls from the media library with set option by specifyng url value of "null" (or null, or "") (requires JSON API v6.9.0+)
  • Add: Support for setting of season fields (requires JSON API v6.10.0+)
  • Add: Support for setting of set (movie set) fields (requires JSON API v6.12.0+)
  • Add: rdirectory option, a recursive version of directory
  • Add: jr and Jr options for "raw" JSON output that isn't guaranteed to be ASCII (urls will be decoded/unquoted, however)
  • Chg: For reasons of consistency, cache.ignore.types patterns no longer need to specify image:// prefix. Defaults are now: ^video, ^music. Old patterns that include ^image:// will be automatically corrected to remove image://.
  • Fix: Reworked charset encoding, hopefully saner than before, now with fewer conversions and increased consistency between Python2 & Python3.
  • Fix: Added some memory optimisations to try and reduce memory consumption on low memory devices (eg. R-Pi)
Update to tools/mklocal.py:
  • Replace --add argument with --artwork
  • Replace --nodownload/-n with --readonly/-r
  • Fix: Foreign encoding in filenames
See this post for a development build of OpenELEC for Raspberry Pi with the JSON v6.12.0 API.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Is it possible to use this on an ATV2 (running XBMC Frodo) to remove all cached images generated by video add-ons only, and nothing else?

I did read it will be required to install python 2.73, and I am fine with that. But before delving further into this, I'd like to know if it is possible and smooth enough. Specific examples would be very helpful. TIA.
Reply
(2013-11-06, 23:06)ctawn Wrote: Is it possible to use this on an ATV2 (running XBMC Frodo)

Possible, yes, though it's often a challenge to find the right sqlite package - there were some notes on how to accomplish this earlier in the thread, I think the OP has a link in the installation section.

With the introduction of the Textures JSON API (Gotham only) it will be possible to make the dependency on sqlite3 entirely optional, so as long as you're running a recent master build you should be able to run the texturecache.py script without sqlite3 being installed. The next version of texturecache.py will support this optionality, v1.0.4 will fail to run if sqlite3 is not installed.

(2013-11-06, 23:06)ctawn Wrote: to remove all cached images generated by video add-ons only, and nothing else?

I did read it will be required to install python 2.73, and I am fine with that. But before delving further into this, I'd like to know if it is possible and smooth enough. Specific examples would be very helpful. TIA.

Sure, if you are able to identify the matching urls for the video addons, as there is nothing in the texture cache database that associates a cached image with the script, addon or library that created (or "owns") it, other than possibly some unique location or web address.

Here's a few BBC iPlayer cached images that I found using the "s" (search) function - note that several columns have been removed for clarity, those shown are id, cachedurl, lasthashcheck and url:
Code:
# ./texturecache.py s iplayer
000002|b/b6a8e02e.png|2013-11-06 05:04:54|/storage/.xbmc/addons/plugin.video.iplayer/icon.png
018704|c/cdeff6b3.png|2013-11-06 22:16:38|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/popular.png
018705|3/3986054e.png|2013-11-06 22:16:40|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/highlights.png
018706|5/5d70a9db.png|2013-11-06 22:16:40|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/tv.png
018707|e/e3bdcbd6.png|2013-11-06 22:16:27|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/categories.png
018708|3/323ff724.png|2013-11-06 22:16:37|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/search.png
018709|0/034b523c.jpg|2013-11-06 22:16:36|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_one.png
018710|3/360222ba.jpg|2013-11-06 22:16:40|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_four.png
018711|d/d3529073.jpg|2013-11-06 22:16:40|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_three.png
018712|b/b9346b1f.jpg|2013-11-06 22:16:44|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/cbbc.png
018713|4/4ba1eeea.jpg|2013-11-06 22:16:44|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/cbeebies.png
018714|2/2f9ffca4.jpg|2013-11-06 22:16:44|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_news24.png
018715|2/2c9085cf.jpg|2013-11-06 22:16:44|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_parliament.png
018716|e/e02302b6.jpg|2013-11-06 22:16:34|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_two.png
018717|a/a59ccda5.jpg|2013-11-06 22:16:44|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_hd.png
018718|c/ce03df8c.jpg|2013-11-06 22:16:44|/storage/.xbmc/addons/plugin.video.iplayer/resources/media/bbc_alba.png
031335|9/926dc24d.jpg|                   |http://www.bbc.co.uk/iplayer/images/episode/p01crz3s_512_288.jpg
031336|a/a3783f88.jpg|                   |http://www.bbc.co.uk/iplayer/images/episode/b03gf54z_512_288.jpg
031337|5/51a719b6.jpg|                   |http://www.bbc.co.uk/iplayer/images/episode/b03gf5k7_512_288.jpg
031338|0/029f2b1a.jpg|                   |http://www.bbc.co.uk/iplayer/images/episode/b03gmh7k_512_288.jpg
031339|7/79057c60.jpg|                   |http://www.bbc.co.uk/iplayer/images/episode/b03gf5xf_512_288.jpg
Matching row ids: 2 18704 18705 18706 18707 18708 18709 18710 18711 18712 18713 18714 18715 18716 18717 18718 31335 31336 31337 31338 31339

The local artwork you probably don't want to delete, but the remote artwork you do, so search using a more complete URL that identifies only the remote artwork:
Code:
./texturecache.py s http://www.bbc.co.uk/iplayer
then delete the matching ids with:
Code:
./texturecache.py d 31335 31336 31337 31338 31339

You can script this:
Code:
IDS=$(./texturecache.py s "http://www.bbc.co.uk/iplayer" 2>&1 1>/dev/null | grep "Matching row ids" | cut -b19-)
[ -n "$IDS" ] && ./texturecache.py d $IDS

then run it as often as you like, maybe under cron.

You could also return just the iplayer artwork which hasn't been hashed (possibly indicative of it being temporary/transient), eg.
Code:
./texturecache.py x 'where lasthashcheck = "" and url like "%iplayer%"'
then delete the returned id's as you would when using the "s" option, however the above query isn't guaranteed to work right now with the new Textures JSON API (fingers crossed an update might change that).

Should you delete any addon artwork that you meant to keep, just re-cache it (./texturecache.py c addons), or leave it to be re-cached automatically as you browse the GUI.

Is that specific enough for you? Wink
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks for this tool, finally I fixed the artwork display for few movies in my library. (I hope XBMC team fix it in future releases)

I have a small issue, not sure if it isn't supported by the script or my fault. Using your example, I did:

Code:
python "c:\folder\texturecache.py" C movies "Dr. No"

Good, now I see the updated disc, clearart, logo, etc. But XBMC still using the old disc art image for collections/sets in library view, check screenshot bellow (Ace skin right list mode):

Image
(number 1, 2, 3 are the old images)

How I can fix it?

Command bellow also doesn't updated the stacked disc art display:

Code:
python "c:\folder\texturecache.py"  C sets

ps: Frodo v12.2

==============

Update:

Okay, checking the reference is database I found both images:

Image

1 = the old one
2 = new

I don't want to edit/replace manually because there is a file with same name, but different size in each folder: c7c13898.dds, c7c13898.dds I don't know how to generate that kind of file. Found this info in Wiki.

If I just delete both file in folder, XBMC will re-cache them? ok I tried here, XBMC isn't loading the image now. Restored.

Okay, I managed to fix the disc art display, for some reason there is two entry for each movie. I edited the field pointing to the correct image:

Image

Your prune feature is amazing, the thumbnails folder decreased 85%! How can XBMC team not implementing an auto prune? In few month it accumulate a lot of garbage.
I wish to have Rotten Tomatoes (user+critics) + IMDB both ratings display in skin.
Reply
(2013-11-07, 11:46)tinybutstrong Wrote: If I just delete both file in folder, XBMC will re-cache them? ok I tried here, XBMC isn't loading the image now. Restored.

Okay, I managed to fix the disc art display, for some reason there is two entry for each movie. I edited the field pointing to the correct image:

Why did you blank out the filenames in your image of the database rows, that might have been useful in determining what is going on as it looks like one discart has a forward slash, the other a backward slash. If you want to send me your Video and Texture databases I can try taking one last look at resolving any remaining issues with mangled filenames, as I suspect that is your problem.

Edit: I've just noticed you also have two each of clearart and logo images, one with a forward slash the other with a backslash, chances are XBMC is still using the "old" clearart/logo images. I can only imagine you've used an addon that has knackered your urls in a similar way to Movie Set Artwork Automator (now fixed) - maybe contact the addon maintainer about that. Send me your databases and I'll see if I can work around it (though thought I had last time, will be interesting to see what is in your MyVideo db).

Edit2: It's a problem with your sets, right? Try running the following:
Code:
python "c:\folder\texturecache.py" C sets "007" @logfile=c:\tc.log @logfile.verbose=yes
and then upload the tc.log somewhere.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks, that's very specific. Especially that whole grep bit. Can you specify a range of IDs? I also wonder about the unhashed bit, too. Could that be a way to purge all the temp images right there? Would it ever get important images, like fan art or posters or TV episodes?
Reply
(2013-11-07, 21:39)ctawn Wrote: Can you specify a range of IDs?

The script doesn't support a range, but does support multiple ids, so you could construct a range of id's in a script and then pass them in as arguments, but I'm not entirely sure why you'd want to do that.

If you really want to delete a specific range, try the query "where id >= 100 and id <= 200" on the x option, and you'll get all the ids between 100 and 200 (if there are any).

(2013-11-07, 21:39)ctawn Wrote: I also wonder about the unhashed bit, too. Could that be a way to purge all the temp images right there? Would it ever get important images, like fan art or posters or TV episodes?

It's possibly a way of identifying temporary artwork, but it's not something I've looked into in order to confirm this behaviour so it may also clobber images you want to keep.

You could find all ids with no hash and delete them using the "x" option, but just be aware that the current JSON API doesn't support filtering by empty string and will return every row with or without a hash so force @dbjson=no for now.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
ok, i installed python 2.73 and your script, and ran a search But i get this message:

Quote:FATAL: The task you wish to perform requires that the JSON-RPC server is enabled and running on the XBMC system you wish to connect.
[…]
A connection cannot be established to the following JSON-RPC server: localhost:9090

What does that mean? I have the remote control options enabled in XBMC. But what do I do about JSON-RPC server?

Also, if I can get that to work, can't I query by the class 'addon' to do what I am looking for (delete all images from add ons)?

Thanks.
Reply
Add "@dbjson=no" to the command line, but for whatever reason it can't connect to the RPC service on your client so check you've specified an xbmc.host. Sometimes the RPC service will crash, so try restarting XBMC.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Missed this off my previous reply.

(2013-11-07, 22:21)ctawn Wrote: Also, if I can get that to work, can't I query by the class 'addon' to do what I am looking for (delete all images from add ons)?

Thanks.

You can, but it will only delete those images the addon has registered in the media library, which is usually just a "thumb". You can see the artwork associated with your addons by running "jd addons", it won't be much.

The vast majority of the artwork that is stored in the thumbnails cache by an addon has no direct relationship back to that addon. The best hope you have is a unique url, as previously mentioned.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2013-11-07, 22:25)MilhouseVH Wrote: Add "@dbjson=no" to the command line, but for whatever reason it can't connect to the RPC service on your client so check you've specified an xbmc.host.

Sorry, where do I specify that?
Reply
(2013-11-08, 18:55)ctawn Wrote: Sorry, where do I specify that?
On the command line, eg:
Code:
./texturecache.py @dbjson=no c movies

or so it becomes a default setting and you don't need to specify it on the command line each time, create a file called texturecache.cfg and add the line:
Code:
dbjson=no

Edit: If you were referring to xbmc.host, since you're running this on ATV2 it should default to localhost using rpc.port 9090 so no need to specify it explicitly.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
ok, thanks for the hand holding here.

I got this now:

Quote:ATV:/bin root# ./texturecache.py @dbjson=no s youtube
FATAL: The task you wish to perform requires read/write file
access to the XBMC sqlite3 Texture Cache database.

The following sqlite3 database could not be opened:
/var/root/.xbmc/userdata/Database/Textures13.db

Check settings in properties file texturecache.cfg
goa:/bin root#

I don't know if this is an issue with the SQlite installation you mentioned on the ATV, or a path issue. On iOS, that file is located here:
/User/Library/Preferences/XBMC/userdata/Database/Textures13.db
Reply
  • 1
  • 28
  • 29
  • 30(current)
  • 31
  • 32
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17