• 1(current)
  • 2
  • 3
  • 4
  • 5
  • 20
Clean up the Thumbnails folder
#1
This tool does not work for XBMC Frodo or greater. Please use the excellent Texture Cache Maintenance Utility


***See 2nd post for my script to clean up unused thumbnails and fanart***

My Thumbnails folder is growing larger and larger. I'm guessing a lot of the images in there are no longer used, as i have a pretty high turn-around rate for my library content.

Is there any standard way to determine which images in the userdata/Thumbnails folder are no longer used? Or some process to clean them out?

If there isn't a standard way, I will attempt to do it myself, but i need some help. I need to determine what all the valid hash's are, so i can determine which ones can be removed. To do this, I need to determine the CRC hash for every item in XBMC's library that can have a thumb/fanart image.

So far I think I know:
  • Movies, episodes, and music videos use the actual path to the file to generate a CRC hash, for example "smb://server/share/videos/movie/Avatar.mkv" = 1c145e0e. Fanart is stored under the fanart subfolder using the same hash.
  • If the video is a multi-part video, the name only (e.g: stack://part1.ext , part2.ext...) is used to generate the CRC (the path string is ignored)
  • Actor thumbs use the "actor" + [actor name] to generate the CRC, for example Jane Lynch = "actorjayne lynch" = 9f309464
  • Album thumbs are (usually) [album]+[artist] and are also stored in the database under the strThumb field.
  • Album/song thumbs may be taken from a folder.jpg image, and may be generated from the folder path or the song path. More info here on album/song thumbs.
  • Artist fanart is generated using the literal artist name, for example "Metallica" = b18ea041.
  • Artist thumbs are generates using "artist"+[artist name], for example Metallica = "artistmetallica" = 0a2d8dbc.
  • Video Bookmark thumbs store their path in the thumbnailImage field in the database. I'm not sure how it is originally generated
  • TV Show seasons use "season" + [path_to_show] + [season_label]. See here for details
  • TV Show banners/fanart use the literal path to the show, for example "smb://server/TV Shows/Lost/". Fanart is stored under the fanart subfolder using the same hash.
I need to figure out:
  • Are there any other image types I'm missing that get stored in the Thumbnails folder?
I plan on deleting all auto-xxxxxxxx.tbn images since they can be recreated automatically if XBMC needs them.

Thanks!
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
#2
Well, I ended up writing my own script that cleans up the thumbnails directory.

I had 31,563 thumbs/fanart before cleaning up.
After cleanup, I have 12,967....Much nicer
I've been exploring XBMC for a while now and have not found any thumbs missing Smile

-------------------------------------------------------------------------
Here is my script. You can use it to clean up your own thumbnail directory:
Download XBMC Thumbnail Cleaner - v1.1 (Dharma Compatible)
Download XBMC Thumbnail Cleaner - v1.2 (Eden Compatible)
***Currently no Frodo compatible version is available or planned***



Details:
  • This script will delete unused thumbnails, fanart and textures in the userdata\Thumbnails directory.
  • It can be run in simulation mode if you'd like to test it.
  • It uses data from XBMC's video/music/texture databases, as well as the JSON-RPC interface to get a list of all thumbs that are still used by the libraries. Any thumbs that are no longer used are deleted.
  • Any auto-xxxxxx thumbnails will be deleted since XBMC will automatically re-create them if they are needed.
  • Written in Java. Tested in a Windows environment.
Disclaimer:
If you are not running in simulation mode, I recommend backing up your Thumbnails directory, in case something goes wrong, you can easily revert.
This has only been tested when XBMC's language is set to English.

-------------------------------------------------------------------------
If you have problems, please paste your log and ThumbCleanerConfig.xml to pastebin.com and reply here with a link to them.
Here is what a normal log would look like for reference: http://pastebin.com/raw.php?i=5cd6WEYf (v1.0 log)

***Note:
If you are not running this on a Windows environment, you must specify the path to this program's directory as a command line parameter when starting.

Cheers,
BradVido88
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
#3
you can use the fileitem.py in my logo downloader script to get hash and path Wink
Reply
#4
ppic Wrote:you can use the fileitem.py in my logo downloader script to get hash and path Wink
Thanks. I'm not developing in python, but could still be useful for cross-checking.

Edit: for anyone who need to quickly check XBMC CRC hashs, you can use this on windows: http://forum.xbmc.org/showthread.php?tid=116861
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
#5
bradvido88 Wrote:Well, I wrote my own script that cleans up the thumbnails/fanart.

I had 31,202 thumbs/fanart before cleaning up.
After cleanup, I have 11,936....Much nicer Smile
I've been exploring XBMC for a while now and have not found any thumbs missing, so I must have properly determined which CRC hash's are still valid and which can be purged...

If there is much interest in the script, i'll post it.

I am definitely interested in your script! I have the same problem as you and my thumbnail directory is getting way too big (over 3gb). Any help would be greatly appreciated.
Reply
#6
Add me to the people looking for a solution to this problem. My Thumbnails folder has grown exponentially over the years. I have upgraded my NAS a few times and I've also renamed a few items. It's too bad XBMC can't do this on its own.
Reply
#7
Yes, come on guys send your Thumb clean codes. I want to use it Smile

@ppic come on write one for us with python also Big Grin
Reply
#8
Star 
Ditto.... speed and space are always on the forefront when I look deep. Trouble is I get lost, end up wandering into plugins.
Reply
#9
looking for the script too. thumbnail directory here is over 8gb.
Reply
#10
OK, I'll clean out the hard-coding and post it when i have a chance.
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
#11
queeup Wrote:Yes, come on guys send your Thumb clean codes. I want to use it Smile

@ppic come on write one for us with python also Big Grin

you're better in python than me here Wink
Reply
#12
No waaay, i think you are more lazy then me Wink
Reply
#13
the simplest way i see here:
make a list of used thumb.
compare for each thumbs found in thumbnail folder if present in the list, if not, then delete it.
this means it have to list all used thumb ( addon icon for example)
at least, except library content thumb, all other used thumb will be regenerated by xbmc.
Reply
#14
As soon as video and music thumbs are transferred to the texture cache I'll work on adding the cleanup side of things.

Others are welcome to work on the cleanup side of things if you like - basically just open the Textures.db file and take a look at what we have - we have last used and use count available, so we can easily purge at various points (and ofcourse they'll be automatically re-cached if used again Smile

Good strategies for purging need to be dreamt up - I'm sure you guys can take care of that. Obviously these apply only to addons, pictures etc. at this point (i.e. not music or video thumbs).

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#15
jmarshall Wrote:As soon as video and music thumbs are transferred to the texture cache I'll work on adding the cleanup side of things.

Others are welcome to work on the cleanup side of things if you like - basically just open the Textures.db file and take a look at what we have - we have last used and use count available, so we can easily purge at various points (and ofcourse they'll be automatically re-cached if used again Smile

Good strategies for purging need to be dreamt up - I'm sure you guys can take care of that. Obviously these apply only to addons, pictures etc. at this point (i.e. not music or video thumbs).

Cheers,
Jonathan

old habits, i forgot about texture.db !
Reply
  • 1(current)
  • 2
  • 3
  • 4
  • 5
  • 20

Logout Mark Read Team Forum Stats Members Help
Clean up the Thumbnails folder4