2014-03-22, 12:14
I wrote a pair of scripts to easy comparison between CPU utilization during playback of different media files; or different CODECs; or different storages. Disclaimer: I don't claim my method is correct, but it looks pretty convincing on paper.
Testing methodology: From 2L high-def demo samples I borrowed the same song (2L-087) at three formats: 24-bit/96 kHz/stereo, 24-bit/96 kHz/multi-channel and 24-bit/192 kHz/stereo; reencoded to ALAC (with XLD) and decompressed to wave using command-line flac. Also took one of ALAC-encoded CD-rips from my collection and made FLAC and wave copy. The results are 16-bit/44,1 kHz/stereo, 24-bit/96 kHz/stereo, 24-bit/96 kHz/multi-channel and 24-bit/192 kHz/stereo in ALAC, FLAC and wave. Twelve songs were copied to USB stick and stick were plugged into the Pi running OpenELEC-RPi.arm-Milhouse-20140320013122-r17977-ge57e1bc.
1st test script -- time-xbmc-music.sh -- will use JSON RPC API to play each song for a selected time interval, then take a snapshot of CPU and I/O utilization of the Pi for a slightly shorter period of time (this way only playback utilization will be captured); this process is repeated for each one of the test songs. Each song is played in full screen with whatever music visualization has been chosen (None at my end). This script *requires* that hostagent is running on the Pi.
2nd script -- process-results.sh -- will process utilization snapshots to output a tab-delimited file, that can be imported in a spreadsheet App.
I used my own project to track utilization on the Pi (more about this below).
The result at my place looks like this and is not at all surprising, apart from quite large difference between FLAC and ALAC:
Metrics columns are (from 2nd): # of interrupts, # of context switches, CPU user, CPU system, CPU idle and CPU wait utilization, I/O reads from USB in kB/s and how much CPU utilization adds decompression compared to wave; all numbers are averages.
IMHO when comparing numbers should be taken into account that XBMC process uses some constant amount of CPU utilization (to drive GUI, remote controls, etc.), so if wave is 80% idle and FLAC is 70% idle, this difference should be read more like "compared to wave, FLAC decompression adds 10% utilization on average".
Both scripts are shell scripts and have been tested on Mac OS X. They should work on any POSIX OS, and they work on Pi itself. Variables, declared at the top of 1st script can be used to customize the test: IP address of Pi to test on, the list of files, their location from Pi's point of view, and playback and monitoring durations.
CPU and I/O utilization on the Pi is monitored with App named hostagent. It will make a copy of CPU, I/O and network utilization every second and can report that data using a built-in web server (listens on port 8080 by default). It adds less than 1% of CPU usage during metrics collection. It is not necessary to specify, that the program must be run on the Pi.
As a bonus, you can also see how to remote control XBMC from a shell script :-)
Testing methodology: From 2L high-def demo samples I borrowed the same song (2L-087) at three formats: 24-bit/96 kHz/stereo, 24-bit/96 kHz/multi-channel and 24-bit/192 kHz/stereo; reencoded to ALAC (with XLD) and decompressed to wave using command-line flac. Also took one of ALAC-encoded CD-rips from my collection and made FLAC and wave copy. The results are 16-bit/44,1 kHz/stereo, 24-bit/96 kHz/stereo, 24-bit/96 kHz/multi-channel and 24-bit/192 kHz/stereo in ALAC, FLAC and wave. Twelve songs were copied to USB stick and stick were plugged into the Pi running OpenELEC-RPi.arm-Milhouse-20140320013122-r17977-ge57e1bc.
1st test script -- time-xbmc-music.sh -- will use JSON RPC API to play each song for a selected time interval, then take a snapshot of CPU and I/O utilization of the Pi for a slightly shorter period of time (this way only playback utilization will be captured); this process is repeated for each one of the test songs. Each song is played in full screen with whatever music visualization has been chosen (None at my end). This script *requires* that hostagent is running on the Pi.
2nd script -- process-results.sh -- will process utilization snapshots to output a tab-delimited file, that can be imported in a spreadsheet App.
I used my own project to track utilization on the Pi (more about this below).
The result at my place looks like this and is not at all surprising, apart from quite large difference between FLAC and ALAC:
Metrics columns are (from 2nd): # of interrupts, # of context switches, CPU user, CPU system, CPU idle and CPU wait utilization, I/O reads from USB in kB/s and how much CPU utilization adds decompression compared to wave; all numbers are averages.
IMHO when comparing numbers should be taken into account that XBMC process uses some constant amount of CPU utilization (to drive GUI, remote controls, etc.), so if wave is 80% idle and FLAC is 70% idle, this difference should be read more like "compared to wave, FLAC decompression adds 10% utilization on average".
Both scripts are shell scripts and have been tested on Mac OS X. They should work on any POSIX OS, and they work on Pi itself. Variables, declared at the top of 1st script can be used to customize the test: IP address of Pi to test on, the list of files, their location from Pi's point of view, and playback and monitoring durations.
CPU and I/O utilization on the Pi is monitored with App named hostagent. It will make a copy of CPU, I/O and network utilization every second and can report that data using a built-in web server (listens on port 8080 by default). It adds less than 1% of CPU usage during metrics collection. It is not necessary to specify, that the program must be run on the Pi.
As a bonus, you can also see how to remote control XBMC from a shell script :-)
