[map1742]

smb picture loading slowed down for me in xbmc builds in first half 2005. a tcpdump showed nothing unusual expect that the xbox was just taking a long time to request more data. still the improved functionality such as the zoom navigation window more then made up for the slow loading.

more recent builds have the same picture slowdown however now the slideshow navigation and zooming displays black screens and i usually end up with a locked up xbox.

frov posted a good description of the problems in january in his posting "slow pictures when zoomed in" ( http://www.xboxmediaplayer.de/cgi-bin....t=18257 )

i am prepared to try debugging this myself and have installed the various support files and built xbmc from the latest cvs. i am starting with the picture loading speed rather than the state machine in guiwindowslideshow.cpp and i am asking for advice on how best to proceed.

my fileserver runs fedora core 4 with gigabit ethernet and i can read files via samba at 20mbytes/sec sustained from my windows xp desktop.

a tcpdump of xbmc loading a jpeg shows a sustained speed of 800kbytes. a 2.4mbyte jpeg photo takes 3 seconds to transfer and this is also the load time in the xbmc log.

11-03-2006 16:46:41 debug loading the current image smb://bundy/pictures/highres/2006/2006_01_30/img_1052.jpg
11-03-2006 16:46:44 debug finished background loading smb://bundy/pictures/highres/2006/2006_01_30/img_1052.jpg
11-03-2006 16:46:44 debug duration: 761

btw this is also the last line in the log as the xbox lockedup up here.

a tcpdump at the server for this file read has about 20 smb reads with most 1 bytes in length and some larger reads (up to 12kbytes in some jpegs). this appears to be the jpeg header being read. the 12kbytes reads complete in well under 2 milliseconds and 8mbytes/sec bulk read speed appears possible.

the actual bulk data read then starts reading in 4096 byte chunks from offset 0. an pattern quickly establishes itself of 4096 byte smb read transactions 2 milliseconds apart with the next 4096 bytes transaction 8 milliseconds later. this is 8kbyte every 10 miliseconds or 800kbytes/sec. i have attached a section of the decode packet capture.

i have seen this type of slow transfer before in systems that used selects with a timeout for asynchronous i/o rather then using signals or interrupts when a file descriptor was ready. idle systems would transfer disk cached files at 200kbytes/sec whilst busy systemswould transfer the files 10 times faster.

a few questions:

1. is smb file i/o from cximage and in fact are all dlls supposed to be buffered? it appears not to be.
2. does the smb file read code block waiting for network data? if yes does it busy-wait or yield the processor? in either case in which source file?
3. why is the cximage read size so small at 4096 bytes and not the 16384 bytes used in the xbmc file code or even larger?

i have followed the code from the cximage fread() calls to dll_fread() and dll_read() in emu_msvcrt.cpp to the xbmc filesystemswitch through to libsamba cli-read() in libsamba/source/libsmb/clireadwrite.c. there appears to be no buffering or caching in this path. i have not managed to follow the path back to where the libsamba read call blocks and yields the processor.

increasing the 4096 byte read size appears to be the initial step in getting the cximage jped smb read speed up. the definition appears to be the "#define input_buf_size 4096" in cximage/jpeg/jdatasrc.c.



time elapsed size source dest protocol info
"17:14:50.716194" "0.007363" "113" "192.168.0.35" "192.168.0.8" "smb" "read andx request 4096 bytes at offset 86016"
"17:14:50.716367" "0.000173" "1358" "192.168.0.8" "192.168.0.35" "smb" "read andx response 4096 bytes"
"17:14:50.716396" "0.000029" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.716418" "0.000022" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.716439" "0.000021" "301" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.716818" "0.000379" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845648 ack=355388836 win=64512 len=0"
"17:14:50.716943" "0.000125" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845648 ack=355390387 win=64265 len=0"
"17:14:50.718228" "0.001285" "113" "192.168.0.35" "192.168.0.8" "smb" "read andx request 4096 bytes at offset 90112"
"17:14:50.718398" "0.000170" "1358" "192.168.0.8" "192.168.0.35" "smb" "read andx response 4096 bytes"
"17:14:50.718426" "0.000028" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.718447" "0.000021" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.718467" "0.000020" "301" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.718849" "0.000382" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845707 ack=355392995 win=64512 len=0"
"17:14:50.718974" "0.000125" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845707 ack=355394546 win=64265 len=0"
"17:14:50.726846" "0.007872" "113" "192.168.0.35" "192.168.0.8" "smb" "read andx request 4096 bytes at offset 94208"
"17:14:50.727023" "0.000177" "1358" "192.168.0.8" "192.168.0.35" "smb" "read andx response 4096 bytes"
"17:14:50.727052" "0.000029" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.727074" "0.000022" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.727096" "0.000022" "301" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.727473" "0.000377" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845766 ack=355397154 win=64512 len=0"
"17:14:50.727598" "0.000125" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845766 ack=355398705 win=64265 len=0"
"17:14:50.728929" "0.001331" "113" "192.168.0.35" "192.168.0.8" "smb" "read andx request 4096 bytes at offset 98304"
"17:14:50.729107" "0.000178" "1358" "192.168.0.8" "192.168.0.35" "smb" "read andx response 4096 bytes"
"17:14:50.729137" "0.000030" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.729159" "0.000022" "1358" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.729180" "0.000021" "301" "192.168.0.8" "192.168.0.35" "nbss" "nbss continuation message"
"17:14:50.729559" "0.000379" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845825 ack=355401313 win=64512 len=0"
"17:14:50.729682" "0.000123" "60" "192.168.0.35" "192.168.0.8" "tcp" "1029 > netbios-ssn [ack] seq=1048845825 ack=355402864 win=64265 len=0"
"17:14:50.737406" "0.007724" "113" "192.168.0.35" "192.168.0.8" "smb" "read andx request 4096 bytes at offset 102400"

[jmarshall]

hi there, and welcome

your analysis is spot on.

there is no buffering in the process at present. the read process from the dll is exactly as you have described.

we do have a couple of file buffering classes available. one is the replacement cbufferedfile which wraps cfile and simply reads in 16k chunks. a while back i replaced cfile with cbufferedfile to test whether it made any difference at all with loading jpgs.

to my surprise it did not. times to load were basically identical.

we also have a threaded file reader available as well (cfilereader that paplayer uses) that reads up to whatever size you wish in a separate thread. i haven't played with this to see if it helps.

as for what happens in cximage itself, it resamples the read scanlines using bicubic resampling to get down to a suitable resolution so it doesn't run out of memory.

you can build the dll to allow debugging from vs .net if you wish to step through the code etc - see the online manual for details, or there is a note in docs/. the source code for cximage is in docs/ as well.

and the state engine in slideshow is very badly designed - my fault completely - the classic case of starting out simple without much planning for what was to come. i haven't had time to do anything about it. if you have time to consider a reasonable design for it, please feel free to write up a design documentation (and even implement it if you have more time that we can build from.

seems to run fine whenever i try it though - perhaps the images i have aren't large enough (3mp camera only)

as for your questions:

1. see above.
2. not sure whether it blocks other threads or not - darkie et al may be able to comment further.
3. no idea - as in all opensource projects, code is tended to be inherited from others in a working or semi-working condition, and if it's not a high priority, it's not really decently reviewed. sounds like a bigger chunksize would be warranted - at the very least it should be able to read a few scanlines of a jpeg at a time.

cheers,
jonathan