Compressed textures for FanArt - testing here with XBMC 2009-10-05 (r23431) or newer

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #46
bleze Wrote:Does source code exist which can convert a JPEG to DSS without the need for SDL? I'd like to implement something running on a Windows Home Server (Windows 2003 Server).

The Windows download of the nVidia tools does come with source code and includes to incorporate the libraries in your own software, as well as command line binaries of the tools for scripting, so hopefully that might help you.
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,228
Joined: Oct 2003
Reputation: 177
Post: #47
The sourcecode that we use in the texture packer is in SVN as well - see MakeDDS. It uses squishlib which (in my experience) gives very slightly better quality than the nvidia tools. I suspect you'd have to measure the difference rather than see it though.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
arco Offline
Donor
Posts: 51
Joined: Jul 2009
Reputation: 0
Post: #48
IceNine Wrote:I'm glad you are reading this thread, arco! It is great to be able to get some info from someone who has done it firsthand. I'm assuming you are doing the DDS conversion just for fanart, not for other thumbnails? Does anyone know if it would work for movie/tv posters as well? Are there any other settings you changed on your system other than adding the <useddsfanart> setting to your advancedsettings.xml file? Also it sounds as if you are leaving the original .tbn files on the XBMC machine along with the new .dds files. Is this the case, and if so why not just delete the .tbn files and have only the .dds files on the XBMC machine?

I'm only using it for fanart, and I think XBMC only supports it for fanart atm. I don't think dds is needed for anything but fanart anyway, since posters and such loads pretty quickly at my end.

Only change I've done to the system is add the <useddsfanart> to advancedsettings.xml. It will then load the dds file instead of the tbn file.

And I have both the tbn and dds files stored in the fanart folder. I'm not that worry about it using disk space, as I have the media files on another computer and only uses the 330's hard drive for the XBMC system itself. Also it's easier if I want to switch back to using the normal method, and not having to rescrape everything.
find quote
arco Offline
Donor
Posts: 51
Joined: Jul 2009
Reputation: 0
Post: #49
I made a couple of videos showing the difference. Used my mobile to record, so quality isn't that great, but should be good enough for you to see.

http://www.youtube.com/watch?v=AaInuGOdLd4
http://www.youtube.com/watch?v=oALFB9MVfcI
find quote
Smenus Offline
Member
Posts: 72
Joined: Nov 2009
Reputation: 0
Location: Bath, UK
Post: #50
I sometimes have 'pop-in' type effects on wall views, such as the one in Xperience.

I was wondering if using .dds instead of the .tbn files would help, or would even work?

All of my movies have a movie.tbn file, which in most cases is a 1000x1500px JPG, although I expect XBMC caches at a lower res than that.

Cheers,
Sean.
find quote
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #51
Well I have finally gotten to the point where I am ready to try this. I wanted to make things easy on myself in the future, so I downloaded and compiled the nvidia texture tools on my ION machine (took a while for me to figure out how to get it running, it doesn't like Karmic, then had to get CUDA support because compressing on the Atom took *forever*), then I wrote a little script that would automatically find any .tbn files in the Fanart folder and convert them to .dds, then move them to a different folder.

I want to keep the originals in case I want to move off of .dds files, but I wanted to move them out of that folder so that whenever I would add anything new, the script would only convert the new ones.

Since I'm really not much of a linux user before this, it was all a large learning experience. If anyone is trying to do this same kind of setup I should be able to help with any problems you might have, I figure I have run into them all by now.

Based on arco's videos, I am looking forward to the performance increase this should get me. I'll keep everyone posted, this could really breathe new life into XBMC on low power ION machines.

As a followup to the start of this thread, have there been any further developments in the XBMC code for integrating .dds support? Any future plans for this that might be realized in the next major release? I know it would be great to have this kind of thing integrated into the core so the fanart and possibly the other thumbnails as well are using the .dds format to put more cycles into the GPU. I know that not all GPU's are anywhere the same performance level, but they are optimized to handle this kind of thing, so decompression of an image should be something a GPU can handle a lot more easily than a CPU.
(This post was last modified: 2010-01-21 22:01 by IceNine.)
find quote
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #52
Well I got all my fanart converted to .dds format and the speed difference is amazing! There is a size increase (the .dds files take up about 4x as much space as the original .tbn files), but the speed increase is more than just noticeable; it's like a whole different machine. I did some cpu usage tests remotely and scrolling through my movies on Aeon65's Multiplex view uses literally about half the cpu power with .dds files than it did with .tbn files. Scrolling through at full speed before pretty much maxed out the cpu, now it goes a little over half usage. Coverflow view works quite a bit better as well, with no thumbnail lag.

I would say that this endeavor of .dds format fanart is more than worth it for anyone running a low cpu-power machine such as an ASRock or Acer Aspire.

EDIT:

Upon closing inspection, it seems that all my .tbn fanart files have been recreated. XBMC still seems to be loading the .dds files; is it expected behavior for the original .tbn files to be recreated?
(This post was last modified: 2010-01-21 23:24 by IceNine.)
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,228
Joined: Oct 2003
Reputation: 177
Post: #53
That's probably expected.

Note that of particular interest is the time to convert from tbn to dds. It was 7 seconds or so on my dev machine for a 1080p image for instance.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #54
Well I did some benchmarks for you this morning on a 1920x1080 jpg, with and without CUDA acceleration on my ION machine.

CUDA Enabled: 10.300 seconds

CUDA Disabled: 701.500 seconds

CUDA Disabled, using -fast option: 22.300 seconds

Obviously if this ever makes it as a full-blown feature in XBMC and the .dds compression is done internally some sort of CUDA acceleration will be needed for machines such as the ION. It would probably benefit anyone with an nVidia card since CUDA should be as fast if not faster than CPU anyway.

Is there anything else I can do on my end that might help this project along? If none of the devs have an ION machine to test on I'd be happy to run some code for you or give you any information I have gotten through this whole endeavor.

Also: If XBMC is recreating the .tbn fanart, even when <useddsfanart> is on, does this mean that if I had .dds files for all my current stuff but I added something new, would XBMC use the .tbn file for the new thing until I made the .dds file or would it act like it couldn't find the fanart at all?

Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
(This post was last modified: 2010-01-22 20:05 by IceNine.)
find quote
arco Offline
Donor
Posts: 51
Joined: Jul 2009
Reputation: 0
Post: #55
IceNine Wrote:CUDA Disabled: 701.500 seconds

That can't be right? :confused2: I think it took like 20 seconds without CUDA when I tested it in Linux on my ION.

Edit: I used the -fast option.

Quote:Also: If XBMC is recreating the .tbn fanart, even when <useddsfanart> is on, does this mean that if I had .dds files for all my current stuff but I added something new, would XBMC use the .tbn file for the new thing until I made the .dds file or would it act like it couldn't find the fanart at all?

The latter, You will just get a white image.

Personally I wouldn't mind waiting a few seconds extra for the converting, and if the devs dont plan on making dds the default, it would be nice if it could be an option. With as it is today the default behavior, and if you wanted to use dds you had to activate it in the settings.
(This post was last modified: 2010-01-22 19:51 by arco.)
find quote
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #56
Ok, here is some more real world usage information I have been able to do today. I saw the feature request in Trac for this, so I am trying to help as much as I can.

1280x720 image compression:

CUDA enabled: 4.640 seconds

CUDA disabled: 327.470 seconds

CUDA disabled, using -fast option: 10.010 seconds

This is the same image used in the 1080 tests, rescaled to 720. I tried multiple images, and complexity and content doesn't seem to change the compression times significantly.

When I am playing a video in XBMC and run some compressions, with CUDA enabled the compression interferes with playback, but with CUDA off the video plays fine.

Keep in mind I am only using the nVidia Texture Tools for these tests (which use squishlib now also, at least in the Linux versions) on my ION machine. I did a test on a Core2 windows machine and got a similar result to yours, so without CUDA or some other kind of acceleration a moderately powerful CPU will be needed to be able to use this. With CUDA turned off compression takes up one thread of one core of the CPU, which is probably why it doesn't interfere with playback. If the tool was multi-threaded compression would probably be faster.

Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
(This post was last modified: 2010-01-22 20:06 by IceNine.)
find quote
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #57
arco Wrote:That can't be right? :confused2: I think it took like 20 seconds without CUDA when I tested it in Linux on my ION.

Edit: I used the -fast option.



The latter, You will just get a white image.

Personally I wouldn't mind waiting a few seconds extra for the converting, and if the devs dont plan on making dds the default, it would be nice if it could be an option. With as it is today the default behavior, and if you wanted to use dds you had to activate it in the settings.

Ha, tried -fast myself and it was as advertised! Went from 327 seconds to 10 on the 720 image. Visually I couldn't tell any difference between the -fast and not.

I'm with you as far as waiting for conversion if this ends up in code. Maybe not as default, but an option would be great. The speed difference in the GUI is huge and conversion would only really take a lot of time if you are adding lots of things at once. I would be more than happy to spend an extra 5-10 seconds importing my fanart if it meant keeping my GUI as snappy as it is now.

I did some more benchmarking using the debug option in XBMC to measure the GUI fps. Fast scrolling through my movies using .tbn files the fps go down to about 10-12, using .dds it is 23-25. Much smoother!

Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
find quote
moontan77 Offline
Fan
Posts: 387
Joined: Jul 2009
Reputation: 0
Post: #58
why was the zip file removed from the first post?
find quote
bleze Offline
Senior Member
Posts: 215
Joined: Nov 2009
Reputation: 0
Post: #59
what about posters/covers. would help on scrolling instead of those placeholders showing up when scrolling fast
find quote
IceNine Offline
Senior Member
Posts: 126
Joined: Jan 2010
Reputation: 0
Post: #60
moontan77 Wrote:why was the zip file removed from the first post?

I'm not sure why it was removed, but you have a couple of options if you want to try this out. I'm personally using the nVidia Texture Tools from here which has Windows and Linux versions (you'll have to compile on Linux though), or if you can compile on Windows the code for the original MakeDDS.exe is in the SVN repo here.

Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
find quote
Post Reply