• 1
  • 3
  • 4
  • 5(current)
  • 6
  • 7
  • 26
Compressed textures for FanArt - testing here with XBMC 2009-10-05 (r23431) or newer
#61
bleze Wrote:what about posters/covers. would help on scrolling instead of those placeholders showing up when scrolling fast

I would like to see this kind of thing as well; using the .dds format for every image possible would most likely make XBMC run much more smoothly on ION machines and the like. It is entirely up to the devs to implement such a thing though.
Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
Reply
#62
how are u guys compressing the whole fanart folder? i can only do one at a time with the nvcompress. Could someone reattach the file that was on the first post please.
Reply
#63
Hi all, i've readed with interest all this thread and what i'm reading is amazing for users (like me) with ION based machines and Atom processors (i've watched both videos in youtube and the performance gain is really good). Currently i'm using XBMC Live (linux) and it could be nice to test this new feature on my Zotac MAG HD-ND01, it could be very very useful if someone (may be in another different thread) could post the steps needed to use this feature (yes, i already know about advancedsettings.xml change) and to create a cron job that converts all images daily (or every x hours). WHere can i download de nVidia CUDA tools to convert the images? how can i create a little script to convert and remove original images in target folder or folders (i'm not sure where must i run this script)

Well, any help would be most welcome, thanks in advance and keep up the good work guys!

kind regards,
Reply
#64
EMK0 Wrote:how are u guys compressing the whole fanart folder? i can only do one at a time with the nvcompress. Could someone reattach the file that was on the first post please.

I am compressing with nvcompress also, on the ION machine that XBMC is running on. I just wrote a little script to automate the process, but that was before I knew the .tbn files would recreate themselves if I moved them so I kinda need to rewrite it to be a bit more elegant. Unfortunately I'm not much a scripter, so if nobody beats me to it it might take a little while. I'll paste in the original script anyway if you want to use or modify it. All it does is rename all the .tbn files in the Video/Fanart folder to .jpg (have to for nvcompress to work), run nvcompress on all of them, then rename them back to .tbn.

EDIT: Don't use this script, I have reworked it later on in the thread to be much more functional.

Code:
#!/bin/sh

rename 's/tbn$/jpg/' ~/.xbmc/userdata/Thumbnails/Video/Fanart/*.tbn
find ~/.xbmc/userdata/Thumbnails/Video/Fanart/ -type f -name '*.jpg' -print0 | xargs -0 -I {} nvcompress {} -bc1 {}.dds
rename 's/jpg$/tbn/' ~/.xbmc/userdata/Thumbnails/Video/Fanart/*.jpg
Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
Reply
#65
Question 
IceNine Wrote:I am compressing with nvcompress also, on the ION machine that XBMC is running on. I just wrote a little script to automate the process, but that was before I knew the .tbn files would recreate themselves if I moved them so I kinda need to rewrite it to be a bit more elegant. Unfortunately I'm not much a scripter, so if nobody beats me to it it might take a little while. I'll paste in the original script anyway if you want to use or modify it. All it does is rename all the .tbn files in the Video/Fanart folder to .jpg (have to for nvcompress to work), run nvcompress on all of them, then rename them back to .tbn.

Code:
#!/bin/sh

rename 's/tbn$/jpg/' ~/.xbmc/userdata/Thumbnails/Video/Fanart/*.tbn
find ~/.xbmc/userdata/Thumbnails/Video/Fanart/ -type f -name '*.jpg' -print0 | xargs -0 -I {} nvcompress {} -bc1 {}.dds
rename 's/jpg$/tbn/' ~/.xbmc/userdata/Thumbnails/Video/Fanart/*.jpg

Many thanks for sharing the script IceNine, but how can i get nvcompress installed on my XBMC machine? i can't see how to get it...Blush
Reply
#66
fidoboy Wrote:Many thanks for sharing the script IceNine, but how can i get nvcompress installed on my XBMC machine? i can't see how to get it...Blush

Well that is a project that took me almost a whole day. Hopefully I can help you out!

First what you will need:
nVidia Texture Tools
CUDA Tools
g++ and gcc versions 4.4 and 4.3
libjpeg and dev files
cmake

Code:
sudo apt-get install g++ g++-4.3 gcc-4.3 libjpeg62-dev cmake
cd ~
wget http://nvidia-texture-tools.googlecode.com/files/nvidia-texture-tools-2.0.7-1.tar.gz
wget http://developer.download.nvidia.com/compute/cuda/2_3/toolkit/cudatoolkit_2.3_linux_32_ubuntu9.04.run
tar -xvzf nvidia-texture-tools-2.0.7-1.tar.gz
(the cuda extension for nvcompress won't compile on 4.4 for some reason, which is what comes with Ubuntu 9.10 Karmic. If you are running 9.04 you're fine.)
CUDA isn't strictly needed, nvcompress will still work pretty quickly with the -fast option, but CUDA still seems fastest. If you don't want to use CUDA, don't install the cuda tools because the texture tools won't compile without some extra work with g++ 4.3.

To install the CUDA tools, in terminal go to the directory where you downloaded the CUDA file I linked and run

Code:
sudo sh cudatoolkit_2.3_linux_32_ubuntu9.04.run

Now you have to add the cuda libraries to your shared libraries by editing a file. I use nano for my text editor, feel free to use whatever you like.

Code:
sudo nano /etc/ld.so.conf

and add this line to the end

Code:
include /usr/local/cuda/lib/

After you save the file, run

Code:
sudo ldconfig -v

to update the libraries.

Assuming you want CUDA acceleration, after you have installed g++ and ran the CUDA tools installer, move to the nvidia-texture-tools directory (wherever you unzipped it to) and run

Code:
./configure


You should see in the list of things that come up that JPEG is found and CUDA is found. Something like this:

Code:
-- Configuring nvidia-texture-tools 2.0.7-1
-- Setting optimal options
--   Processor: i686
--   Compiler Flags:  -march=i686
-- Looking for OpenGL - found
-- Looking for GLUT - found
-- Looking for DirectX - not found
-- Looking for GLEW - not found
-- Looking for Cg - not found
-- Looking for CUDA - found
-- Looking for Maya - not found
-- Looking for JPEG - found
-- Looking for PNG - not found
-- Looking for TIFF - not found
-- Looking for OpenEXR - not found
-- Use thread library: -lpthread
-- Configuring done
-- Generating done
-- Build files have been written to: /home/xbox/Downloads/nvidia-texture-tools/build

-e Your configure completed 033[92msuccessfully033[0m, now type 033[1mmake033[0m

If that works well, and you are on Ubuntu Karmic then you need to set up g++ to compile using version 4.3.

Code:
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 44 --slave /usr/bin/g++ g++ /usr/bin/g++-4.4 --slave /usr/bin/gcov gcov /usr/bin/gcov-4.4
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 43 --slave /usr/bin/g++ g++ /usr/bin/g++-4.3 --slave /usr/bin/gcov gcov /usr/bin/gcov-4.3

You can confirm that this finished right by running
Code:
sudo update-alternatives --display gcc

Now set the system to run version 4.3 instead of 4.4.

MAKE SURE TO CHANGE THIS BACK WHEN YOU ARE DONE.

Code:
sudo update-alternatives --config gcc

You should see something like this:

Code:
There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path              Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gcc-4.4   44        auto mode
  1            /usr/bin/gcc-4.3   43        manual mode
  2            /usr/bin/gcc-4.4   44        manual mode

Press enter to keep the current choice[*], or type selection number:

In this case you would type 1 and hit enter. Just put whatever number you have to switch to version 4.3.

After this is done, in the nvidia-texture-tools folder, type

Code:
sudo make

and, assuming everything compiled ok (I got some warnings in the compile process but it did finish and everything seems to work fine) type

Code:
sudo make install

and it should install the files properly.

At this point you can change your default compiler back to version 4.4 by running the same command as above and set back to auto (option 0 in the example).

Try to run nvcompress (you should be able to do this anywhere in the command line now) and if you get the typical list of options than you are good to go. Initially I had some issues with nvcompress not seeing the needed shared libraries. You'll know if you have this same problem if, when you try to run nvcompress, you get an error saying it can't find some shared library. If this is the case, follow the next instructions.

Move to the /usr/local/bin directory and check if nvcompress is really not seeing libraries.

Code:
cd /usr/local/bin
ldd nvcompress

You should see a list of libraries that nvcompress depends on. If you have any entries that say Not Found at the end than some libraries aren't being seen. I was missing one (libcudart.so.2) at the end of all these steps, but I fixed this by copying the file to a library I knew was working.

Code:
sudo cp /usr/local/cuda/lib/libcudart.so.2 /usr/local/lib/libcudart.so.2
sudo ldconfig -v

You only need to do something like this if nvcompress gives an error that it can't find a shared library.

Hopefully this should be thorough enough and I didn't forget anything. Pretty much all of this info I found digging around on Google. If anyone runs into any problems that I didn't cover here I can try to help, but I'm not much of a linux guru.
Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
Reply
#67
Here is a new script that I scraped together today that will work much better for how DDS is used, at least for the time being. This script will scan your Fanart folder (currently only for Video fanart, it would be simple enough to add Music fanart if you use that), find any new .tbn files that don't have a .dds file also and convert them. This way you will always have both .tbn and .dds versions of any fanart. It will make things much easier if you switch between the two modes and XBMC won't always be remaking the .tbn files.

Code:
#!/bin/sh

rename 's/tbn$/jpg/' ~/.xbmc/userdata/Thumbnails/Video/Fanart/*.tbn
find ~/.xbmc/userdata/Thumbnails/Video/Fanart -type f -name '*.jpg' -printf '%f\n' | sed 's/.jpg$/\n/' > ~/.xbmc/userdata/Thumbnails/Video/Fanart/found.txt
find ~/.xbmc/userdata/Thumbnails/Video/Fanart -type f -name '*.dds' -printf '%f\n' | sed 's/.dds$/\n/' >> ~/.xbmc/userdata/Thumbnails/Video/Fanart/found.txt
cat ~/.xbmc/userdata/Thumbnails/Video/Fanart/found.txt | sort | uniq -u > ~/.xbmc/userdata/Thumbnails/Video/Fanart/new.txt
cat ~/.xbmc/userdata/Thumbnails/Video/Fanart/new.txt | xargs -I {} nvcompress -bc1 ~/.xbmc/userdata/Thumbnails/Video/Fanart/{}.jpg
rename 's/jpg$/tbn/' ~/.xbmc/userdata/Thumbnails/Video/Fanart/*.jpg
rm ~/.xbmc/userdata/Thumbnails/Video/Fanart/found.txt
rm ~/.xbmc/userdata/Thumbnails/Video/Fanart/new.txt
Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
Reply
#68
Thumbs Up 
WOW! Great tutorial IceNine, it's really useful, many thanks! i'll try it ASAP. I hope not to have any problem (i'm using karmic also).

kind regards,
Reply
#69
Question 
Hi again IceNine, i'm trying to get this working right now, but i've problems when using "sudo make" into "nvidia-texture-tools" folder, it gives me an error saying that target is not specified and there isn't any makefile.

What i'm doing wrong?
Reply
#70
fidoboy Wrote:Hi again IceNine, i'm trying to get this working right now, but i've problems when using "sudo make" into "nvidia-texture-tools" folder, it gives me an error saying that target is not specified and there isn't any makefile.

What i'm doing wrong?

Did you run configure first?

Code:
sudo ./configure

@IceNine, thanks for the script! I'll try it out on my box later. Smile
Reply
#71
fidoboy Wrote:Hi again IceNine, i'm trying to get this working right now, but i've problems when using "sudo make" into "nvidia-texture-tools" folder, it gives me an error saying that target is not specified and there isn't any makefile.

What i'm doing wrong?

Now I knew I was going to forget something. Huh I think the problem is that you need to install cmake along with the other dependencies to compile this.

EDIT: I tried to compile the nvidia tools on a fresh Ubuntu 9.10 install and have updated the instructions so they actually work now. You should be able to follow them to the letter and it will work.
Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
Reply
#72
Very interesting thread.

Has anyone attempted to use the dds on the osx/appletv?
Reply
#73
Thumbs Up 
arco Wrote:Did you run configure first?

Code:
sudo ./configure

@IceNine, thanks for the script! I'll try it out on my box later. Smile

Blush You are right arco, i was so excited with this that i forgot that line! Now all is working like a charm, i've also created a cron job to automate the process every day... I'll report any extrange issue here. Again, big thanks to IceNine for his effort...

Another little question, can i uninstall gcc++4.3 and other libraries after doing this process? do they affect to my comp performance if i keep them installed?

kind regards, Big Grin
Reply
#74
Feel free to uninstall the libraries in my instructions, they aren't really needed (unless you compile something in the future, then you might need them again). There really isn't any problem leaving them installed, the only thing they will do to your machine is take up a few MB of hard drive space.
Client: XBMCuntu Frodo - ASRock ION 330 Pro - Logitech Harmony One
Server: 4U NORCO RPC-4220 20-drive case - UnRAID 5.0 - 38 TB parity-protected storage
Reply
#75
Wow, the difference with dds fanart is amazing, great work! Smile

Here's a script creates dds files for any tbn files that don't have one already.
It doesn't rename all the files like the other scripts and it has a progress indicator.
Code:
#!/bin/sh
cd ~/.xbmc/userdata/Thumbnails/Video/Fanart/ || exit
find . -name '*.tbn' -or -name '*.dds' | cut -c-10 | sort | uniq -u |
while read n; do
mv $n.tbn $n.jpg
nvcompress -bc1 $n.jpg | awk '/taken:/{printf $3}'
mv $n.jpg $n.tbn
echo "s $(ls *.dds | wc -l)/$(ls *.tbn | wc -l)"
done

I seriously can't tell anymore if a fanart is loaded from the 2 sec cache or not. That was the only little annoyance I had with with XBMC. Again, wonderful work! XBMC is now perfect! Smile
Reply
  • 1
  • 3
  • 4
  • 5(current)
  • 6
  • 7
  • 26

Logout Mark Read Team Forum Stats Members Help
Compressed textures for FanArt - testing here with XBMC 2009-10-05 (r23431) or newer1