[smst]

Hi there,

I've implemented a small feature to enable the use of wildcards in thumbnail names. It's currently running on my own Xbox for testing, but I'd like to submit the patch. Before I do so I'd like to solicit opinions on the feature, and ask an API question.

The itch I'm trying to scratch with this feature relates to a "now playing" type of folder: somewhere that holds current episodes of shows for a few days before I watch them and delete them. The contents are therefore fairly dynamic and the individual files require different thumbnails. Adding a new thumbnail for each new file is onerous, and since my computer is here to do things I can't be bothered to do, I want XBMC to work out which thumbnail is appropriate for me.

I have a subfolder (alongside the video files themselves) called "wildcards.tbn". (This is the magic directory that enables the behaviour.) It contains tbn files which may have wildcard characters (I use ~, the tilde) in their names. A tilde means "0 or more characters here", and therefore "Consolevania~.tbn" would be expected to match against, say, "Consolevania 208 twoLate.avi". (A run of N+1 tildes means "N or more characters here"; that's as complex as I've let it get.)

I've modified 'FileItem::GetTBNFile'. After trying the usual thumbnail names, if the fallback "VideoFileName.tbn" doesn't exist we now look for "wildcards.tbn". If that subfolder isn't there, we do nothing more -- users without the magic folder are unaffected by this change beyond that first test. If the folder is there, we consider the contents and try to match each tbn pattern against the target file. I thought about using a regular expression for this but it's really overkill (to escape and transform each filename, then run the regex) when a custom algorithm based on a simple string split does just as well.

Questions:

1. (An API question.) Is 'CDirectory::GetDirectory' guaranteed to give me a listing sorted by name? It's helpful if that's the case, as tbn files starting with the wildcard will therefore appear last, and if "~.tbn" exists as a custom fallback (or "~.avi.tbn" and "~.mpg.tbn" for type-specific fallbacks) it won't be prioritised over a more specific pattern which matches the start of the name.

2. Any suggestions for a better magic folder name? I wanted to use ".wildcards.tbn" (with a leading dot), but Windows Explorer doesn't like letting me create such a name. I can do it at a command prompt of course, but creation of the magic folder should be easy for everyone. I chose to use the tbn extension for consistency with thumbnail fails, and to reduce the likelihood that anybody would have a real folder with this name.

3. Can the magic folder be hidden if it exists? I don't know how to do that... presumably there's a point where 'GetDirectory' is called with a list of appropriate file extensions, and it would have to be removed after that point. (being a folder, the extensions mask presumably doesn't apply). Can somebody tell me where to look? _Should_ the folder be hidden? The only reason I can think of not to is that it might legitimately be a folder containing media -- hence my earlier question about a good name. If the user has created it with the intention of using this feature, he surely doesn't need to see it through the xbmc interface.

4. Am I able to post the patch here for review? (Can I attach it to this post? Is there a bbcode tag for quoting code which doesn't throw bits away?) I haven't coded C in a few years (I had to type the algorithm out in Python and translate it line by line while my brain got used to translating thought into C code) so I may be doing something sub-optimal, or using a poor coding style. (At least I know that the code works!) I'd really appreciate feedback and advice!


A note on performance: as you'd expect, this is slower than the usual thumbnail code. That's one of the reasons that it's "opt-in", by way of the magic subfolder. (Another reason for the subfolder is ease of thumbnail management: they're all in one place and don't get in the way of the actual media files when using a PC to browse.) Without the magic folder, this shouldn't slow anybody down. With the behaviour enabled for the first time, there's a brief pause as all of the thumbnails are found. Re-visiting a folder is much quicker since each thumbnail has now been cached. Thumbnails still have to be found (but just once, of course) when new media is added. For day-to-day use, the biggest impact will probably be made by those media files which aren't matched by a wildcard thumbnail, since we'll check all the thumbnails for them every time. This can be mitigated by either using a catch-all default ("~.tbn" to match anything) which will be cached, or by maintaining an up-to-date set of thumbnails for all the series likely to be passing through the folder.

Does anybody have any comments, suggestions or objections?

Thanks,
Steve

[jmarshall]

Hi there. Thanks for taking the time to do up a patch.

1. No.

2. No preference.

3. That's simple to filter out. It'd be best to probably do it in the media window I should think, but obviously there's a linear performance hit to do it (nothing onerous)

4. You can post a patch here, or via SourceForge. SourceForge is better as it's easier to track. You can always link back to this post (and from this post to there) to save you reproducing the above.

As for performance, the hit isn't too bad (the full directory contents should be cached in GetDirectory() so that CFile::Exists() within the folder should be log(n) time, though obviously we need to check that that is the case)

I personally don't really see the need for this - IMO a list of files is just as useful in this case. Thumbs tend to confuse the picture more when it's a bunch of different shows or whatever. I think this is better attacked by doing a library-style approach where once TV items are scanned in (see the videodb branch) we can offer to continually scan the content and update the library with new items. As TV shows in particular are reasonably easy to match on filename (as you are doing with the thumbs) we can get a much richer representation.

With that said, I don't make the decision, the team does. Thanks again for the patch - it will be considered.

Cheers,
Jonathan

[smst]

Thanks very much for your constructive response. I've uploaded the patch; it's patch 1669982.

I'd already worked around the possibility of GetDirectory not being guaranteed-alphabetical, so that was fine; I was hoping to remove the workaround but it's not a problem. I haven't changed anything to filter out the magic folder from the listing yet; I'll see if the behaviour itself gains any traction first.

Another note on performance: since I now check for the existence of (the traditional) "<filename>.tbn" before "wildcards.tbn", there are two Exists calls where before there were none. Two checks instead of one doesn't change it from being log(n) but, y'know, it's still twice as many. The behaviour doesn't really rely on that second Exists call, but it seemed sensible to allow "<filename>.tbn" alongside the media file to supersede any less specific wildcard match.

As to its utility (or not!) -- I wasn't aware of the videodb branch. Is there somewhere I can read about it? That may well do everything I want, so my own need may be served.

I personally don't really see the need for this - IMO a list of files is just as useful in this case. Thumbs tend to confuse the picture more when it's a bunch of different shows or whatever.

Assuming I understand what you're saying, I disagree about the confusion (for my own tastes). I use List mode to see the file names (a nice, wide name so I can see the show and episode name), so the thumbnails as a part of the list are just small blobs of colour. But I've modified the skin to show the thumbnail a bit larger in unused space on the left of the screen (borrowed that from the PDM skin), so when a show is highlighted I get a nice view of it. It's really about prettifying the view, rather than being a substitute for decent metadata (which I'd imagine the videodb branch is all about).

When there are different shows in the list, the slightly different blobs of colour actually help to group episodes of the same show together; it's easier to grok than by scanning the text.

With that said, I don't make the decision, the team does. Thanks again for the patch - it will be considered.

Understood, and thanks for taking the time to respond.

Cheers,
Steve

[Asteron]

I don't really like the idea of having a subfolder for thumbnails as it will clutter up the view. Maybe if you had the fallback thumbnails in a subfolder in userdata that gets read at startup (so you dont have to check files constantly).

I also tend to watch downloaded video out of an "incoming folder". Things don't necessarily live very long in there so I think using the library would be overkill and like this alternative.

I personally don't really see the need for this - IMO a list of files is just as useful in this case. Thumbs tend to confuse the picture more when it's a bunch of different shows or whatever. I think this is better attacked by doing a library-style approach where once TV items are scanned in (see the videodb branch) we can offer to continually scan the content and update the library with new items. As TV shows in particular are reasonably easy to match on filename (as you are doing with the thumbs) we can get a much richer representation.

I havent been able to use the library without either renaming the files more cleanly or modifying the lookup. Either way this is O(n) work for me which is why I dont do it very often. Thumbnail rules are O(1) (or rather O(m) where m is the number of shows I watch). Really thumbnails do more to make the interface look pretty than database fields do to make things 'richer'.

[smst]

I don't really like the idea of having a subfolder for thumbnails as it will clutter up the view. Maybe if you had the fallback thumbnails in a subfolder in userdata that gets read at startup (so you dont have to check files constantly).

I intend to remove the subfolder from the view -- the idea would be that the magic folder name would be spotted and hidden. I just haven't had time to follow jmarshall's pointers and do it!

Your alternative is an interesting suggestion, but there are a couple of reasons I see not to do it. One is that it forces the wildcard behaviour everywhere, instead of just in one location; I see this as behaviour applicable to the one special folder, where a certain context can be assumed for names. To put it another way: if I have episodes of Foobar and a thumbnail for same, I don't want to use the thumbnail for an film named Foobar -- or for an album by Foobar (for which I've been unable to find specific album art), come to think of it.

Secondly, would the performance saving really be that great? For media files without a thumbnail we'd still need to iterate over the magic folder's contents when processing each media file. Although we could cache the array corresponding to each thumbnail's name when split by the wildcard, which would help a little.

Another idea, which avoids the problem of choosing a good magic folder name: would it be possible to support a new tag inside the '<thumbnail>' tag of a sources.xml entry? Say, a '<wildcards>' tag, naming a folder containing appropriate thumbnails. The behaviour would only be enabled when the tag exists. Advantages: easier to justify hiding the thumbnail folder, or to put the thumbnail folder outside of the current directory; allows one thumbnail folder to be used for multiple sources without applying it to _all_ sources; could work for subfolders of a source (I guess -- I presume I could interrogate the parent source's options when inside a subfolder). Disadvantages: requires editing sources.xml, or the addition of a GUI to set the option; requires that a folder with the magic behaviour is a top-level source, rather than just any old folder (although my "now playing" directory is used so much it _is_ a top-level source, which I suspect is common).

What does anybody think about this last suggestion? Can somebody point me towards a starting place in the code?

I also tend to watch downloaded video out of an "incoming folder". Things don't necessarily live very long in there so I think using the library would be overkill and like this alternative.

Good to know I'm not the only one with a use for this.

I havent been able to use the library without either renaming the files more cleanly or modifying the lookup. Either way this is O(n) work for me which is why I dont do it very often. Thumbnail rules are O(1) (or rather O(m) where m is the number of shows I watch). Really thumbnails do more to make the interface look pretty than database fields do to make things 'richer'.

What you say about thumbnail rules is true, and is what motivated the change. Yesterday I started watching a new show, and downloaded a thumbnail for it; next week I won't need to worry about that.

[smst]

Nearly a year later, I return to this thread to ask for a little help. I'd really like to get this patch accepted (for the somewhat selfish reason that I'd like to decommission the Windows box I have which is the only one here that can compile XBMC!), but it hasn't gained much traction.

I guess that the problem is that the new behaviour isn't completely transparent: either the 'wildcards.tbn' directory is displayed (which adds an unnecessary item in the listing for those who use this patch), or it's not (which could hide a legitimate directory, although the chances are very small that somebody will have one with that name).

As I mentioned before, there are two possible solutions:

1. Hide 'wildcards.tbn' whenever it shows up in a listing (and therefore assume that nobody will really create a directory with that name; we could of course mandate a different magic name in the code if we like).

2. Use sources.xml to define a wildcard thumbnails directory for a given source.

I presume that (1) is much easier to implement, but I'm not sure I'll be able to do it or where in the code to start. Could anybody give me a concrete pointer to what to do, please? (I also need to discover how to make a Windows build now, since it's been months since I successfully managed it.)

(BTW, I did try the new library functionality but it was unfortunately not suitable for my needs. It seems geared to a directory structure of archived episodes, rather than the flat list of a few current episodes that I'm dealing with. The feature looks excellent for those purposes and I look forward to using it, but for automatically prettifying a list of files with a simple thumbnail it's not appropriate.)

Thanks in advance for any help.