diyAudio DSP plugin
#16
I'm also a member of the diyAudio forum but haven't been involved in the threads that stef_tsf has mentioned here, though I am enthusiastic about audio DSP and have been using xbmc (more recently openelec) for years for my main system and see no reason that would change in the foreseeable future. Shoehorning all of the audio playback DSP into a HTPC application leaves you with one source only for AV content ... I'd rather xbmc devs stick to their current plans, I've looked and they seem to fit everything that any sane person would want.



Having said that I have seen that there are some challenges with integrating audio DSP device to HTPC. Here are my thoughts, when I have time myself I'll be looking at these.

- jsonrpc api allow announce of audio stream details (bitrate and bit depth)
- jsonrpc api allow access to audio stream delay setting

I checked if this was possible earlier this year and I know there was some overhaul of this section of the code so I left it alone till things had calmed down in that area.

This would allow for things where the audio processing delay is long and impacts on AV synch but the delay is different for audio data of different rates (this happens a bit, I'll leave the boring details out)

With that all video content delay adjustments could be managed via jsonrpc from a network connected audio DSP device ... with minimal code changes (and maintenance) necessary on xbmc end.

I imagine that the start of playback with a device like this would require a setup procedure similar to the way xbmc currently switches display to native refreshrate on initialisation of the video playback, except that the communication would be via jsonrpc rather than display devices.
Reply
#17
I still think that the functionality you need can be integrated either in a sink (mostly not that optimal as we need to output the data later) or as a in between stage, for example after we have "decoded" the audio to float. At this point the audio is not touched at all and you can "have" whatever you want, all channels, we can do the downmixing for you, so you get 2 or whatever channels you want. From here you can do what you find appropriate, just manage your part of GetDelay and afterwards we take over and put out your altered sound.
First decide what functions / features you expect from a system. Then decide for the hardware. Don't waste your money on crap.
Reply
#18
(2013-11-23, 11:41)fritsch Wrote: I still think that the functionality you need can be integrated as a in between stage, for example after we have "decoded" the audio to float. From here you can do what you find appropriate, and afterwards we take over and put out your altered sound.
Agree. Such "in between stage" would be a brilliant novelty, serving many purposes in a well structured way.
Let me sketch the "deluxe" implementation basing on four dedicated stacked "in between stages" of that sort.

There would be a new field into XBMC Audio Configuration, labelled "MIX", naming the third-party DSP module to be loaded and executed for implementing the downmix. This can be a - none - downmix for preserving the integrity of the source. This can be a stereo downmix for always outputting a left and a right channel. This can be a 4-ch downmix also. Nothing prevents fancy things like downmixing in stereo, then applying effects and outputting 8 audio channels. XBMC won't be involved into such complexity. XBMC is only enabling such "in between stage". Bear in mind that within the MIX module, we act as M channel sink, and N channel source. By default, upon a fresh XBMC install, the name of the MIX module to be loaded and executed would be - none - for retrocompatibility.

There would be a new field into XBMC Audio Configuration, labelled "RES", naming the third-party DSP module to be loaded and executed for implementing a high quality resampler. For maintream systems, this will be a 48 kHz resampler. For high-end systems, this will be a 96 kHz resampler. The resampler is supposed to solve sync issues like the incoming audio being not related to the HDMI clock, or being not related to the USB clock. Other sync strategies can be implemented like asynchronous audio. Over there you'll see passionate debates and various strategies, more or less accepted by audio enthusiasts. Anyway, XBMC doesn't see the complexity of this. Within the RES module, we still act as N channel sink, and N channel source. By default, upon a fresh XBMC install, the name of the RES module to be loaded and executed would be - none - for retrocompatibility.

There would be a new field into XBMC Audio Configuration, labelled "PRE", naming the third-party DSP module to be loaded and executed for implementing a pre-processor. This can be a IIR-filter based equalizer, a FIR-based equalizer, a combination of the two, the settings may differ from channel to channel, and there could be a dynamics compressor also, and a room decorrelator also. Within the PRE module, we still act as N channel sink, and N channel source. By default, upon a fresh XBMC install, the name of the PRE module to be loaded and executed would be - none - for retrocompatibility.

There would be a new field into XBMC Audio Configuration, labelled "XO", naming the third-party DSP module to be loaded and executed for implementing a speaker crossover. This can be a IIR-filter based crossover, a FIR-based crossover, a combination of the two. XBMC won't be involved into such complexity. XBMC is only enabling such "in between stage". Bear in mind that within the XO module, we act as N channel sink, and Z channel source. For a stereo crossover we'll have N = 2 and Z = 8. As a footnote I'll say that some experimenters will try applying the crossover concept in a multichannel context. There may be crossovers with N = 4 and Z = 8. This way all four loudspeakers can become active 2-way. This way the front left and front right can become active 3-way, with the rear left and rear right speakers remaining what they are. No centre channel, no separate subwoofer. By default, upon a fresh XBMC install, the name of the XO module to be loaded and executed would be - none - for retrocompatibility.

I'm apologizing by advance for such long post.
I feel it was needed, for clarifying the long term goals, and for suggesting a practical implementation matching fritsch generous proposal.
Please let me know.
Reply
#19
XBMC is a complex application with an architecture we have to maintain. There is a mechanism how to integrate 3rd party addons and we certainly won't violate the rules here.
Audio engine has an architecture as well with a processing stage. Moving audio processing into a sink violates the rules of this architecture.
What we currently have is flexible enough to do this. No need for any quick hacks.
Reply
#20
If a DSP interface is introduced then in my view we shouldn't be polluting the standard audio settings GUI with what is a specialised function that will likely have a limited userbase.

Would it be possible to have a multiplatform jack audio server sink which hands audio over to an external program so the configuration & processing is done entirely outside of XBMC? or if anything was needed within XBMC, could that be done via an add-on with a new audio extension point which could then interface to the external DSP for configuration?
Reply
#21
Quote:Would it be possible to have a multiplatform jack audio server sink which hands audio over to an external program so the configuration & processing is done entirely outside of XBMC?

If you mean a sink in terms of AE sinks, no. At least not in a clean way. Our sink stage is designed just to feed audio packets into the drivers. Doing processing here would raise the risk of underruns in the hw buffers or in other words: we would go back to what we had with SoftAE. I don't think this is desired.

Quote: or if anything was needed within XBMC, could that be done via an add-on with a new audio extension point which could then interface to the external DSP for configuration?

Absolutely. We can define the extension points where in the processing and configuration stages external DSP functions can be hooked in. Every addon has its configuration dialog and we also can provide menu hooks for bringing up windows owned by the addons.
Reply
#22
Please consider the above post as exceptional, a kind of template for illustrating what's going on, to newcomers. Sorry for the length!
Indeed, this present XBMC discussion can't host debates about what's diyAudio, what's a downmixer, what's a resampler, what's a dynamics processor, what's a room decorrelator, what's a loudspeaker crossover, etc.
On diyAudio you can find more about such template, and leave extensive comments or suggestions about it.
Check http://www.diyaudio.com/forums/pc-based/...ost3713825
Reply
#23
I didn't mean any processing would be done in the sink.

I thought Jack was just an interface that provides a virtual audio device for linking audio programs together, so a Jack sink would just pass the audio in it's original format with no processing applied then it would be up to the external Jack server & DSP to do the processing and return it to XBMC via the Jack interface.

I could of course be talking utter bollocks as it's not something I've much knowledge on.
Reply
#24
(2013-11-23, 14:53)jjd-uk Wrote: If a DSP interface is introduced then in my view we shouldn't be polluting the standard audio settings GUI with what is a specialised function that will likely have a limited userbase.
I'm only advocating for 4 fields in the XBMC Audio Configuration panel, explicitly naming who is in charge of what in the audio pipeline : MIX, RES, PRE, XO. Failure to properly represent the audio processing pipeline will prevent XBMC becoming a trustable high-end audio platform, and will prevent XBMC becoming a major (and positive) discussion theme in diyAudio. Bear in mind you can default those 4 fields as "none" or as "xbmc" if you prefear. This is really simple and intuitive. Your audience will prefer loking at this, than anything else, or than being kept in ignorance. The real difficulty with the XBMC Audio Configuration panel - just like any multimedia player - is where one need to name the hardware peripheral to be selected: motherboard (analog jacks, S/PDIF, HDMI), USB audio attachment, or anything else. It gets really complicated because each peripheral needs to expose the features (motherboard has many, and redundant), and on top of this comes the question of eventually sending the audio over multiple peripherals. Don't overcomplicate things. Don't think in terms of a "DSP interface". Be honest. Realize XBMC is already, massively a "DSP" thing. Try being becoming explicit about the audio processing pipeline, and everybody will feel happy. That's my best recommendation. It will be your decision, and your responsibility.
Reply
#25
The Audiophiles who want DSP are an extremely small minority, probably less than 1% of our userbase, thus I don't want to see anything that might confuse the user who couldn't care less about DSP. This is something that needs to be done outside of the current settings, maybe a new screen or preferably via an Add-On in my view.

Once code is in the core app then someone has to maintain it, in my view all we should do is provide the API and hooks into ActiveAE then the actual processing is done by an Add-On or external app, that way there is minimal code to maintain. It's then up to the audiophile's to either write & maintain an Add-On or hook an external DSP into XBMC's API.
Reply
#26
Again too much Text. An interface on howto add or replace a specific stage is enough. If you don't like the Sink, implement another.

Configuration of your stage can be done by an addon of yours or whatever.
First decide what functions / features you expect from a system. Then decide for the hardware. Don't waste your money on crap.
Reply
#27
(2013-11-23, 16:51)jjd-uk Wrote: I don't want to see anything that might confuse the user who couldn't care less about DSP.
Quite disappointing but I can live with that. You are still using the ancient old school vocabulary about "DSP" just like XBMC wasn't a whole "DSP" monster. There are still many people like you nowadays. Probably XBMC is relying on ffmpeg as "blackbox" causing an ugly trivial difficulty when comes the moment of explicitly representing the whole audio processing pipeline. Don't worry, diyAudio enthusiasts know how to deal with that. Basically XBMC would deliver the stereo downmix issued by ffmpeg, to a third-party app acting as stereo audio sink.
Knowing this, in charge of XBMC UI, you need to chose between two different implementations.
Implementation 1 : after delivering the stereo downmix to a Jack sink, XBMC isn't involved in audio anymore.
Implementation 2 : after delivering the stereo downmix to a Jack sink, XBMC queries a Jack source (having 8 channels), and thus, XBMC stays in charge of delivering the audio to the hardware peripheral that's selected by XBMC, preferably the HDMI multichannel LPCM.
Bear in mind that an acclaimed method, would be to establish a bridge ffmpeg -> ASIO, for delivering the ffmpeg downmix under ASIO. This way, any .exe compiled by Flowstone would read the stereo ASIO stream, process it, and deliver another ASIO stream (now 8 channels) to be sent to HDMI multichannel LPCM.
Reply
#28
ffmpeg is no blackbox, you don't need to trust it, it is open source, you can exactly read what it is doing.
First decide what functions / features you expect from a system. Then decide for the hardware. Don't waste your money on crap.
Reply
#29
i would like to see some paragraphs in the text for easier reading. it's still all just a blob of text.
Read/follow the forum rules.
For troubleshooting and bug reporting, read this first
Interested in seeing some YouTube videos about Kodi? Go here and subscribe
Reply
#30
(2013-11-23, 20:33)fritsch Wrote: ffmpeg is no blackbox, you don't need to trust it, it is open source, you can exactly read what it is doing.
Depends what you call a blackbox. One one hand, a blackbox is something you need to take "as is". On the other hand, a blackbox has specifications.

As I wrote above (you may have not realized), diyAudio can live with that and is more than happy with ffmpeg because ffmpeg has parameters (that eventually can be adjusted) and because the ffmpeg source code can be audited. Don't worry, your suggestion remains much appreciated, and I thank you for that.

1. How to tell XBMC to select a custom audio sink instead of the standard XBMC audio sink? Adding one SINK field in the XBMC Audio Configuration Panel? By default the SINK field would read "xbmc". In case there are custom audio sinks registered, the user would see their names, and the user could select one. Right?

2. Say the user wants to output the image (or movie) on HDMI, along with the custom processed sound. How to do this?
Your proposal consisting of an "in between" stage becomes even more valuable.
In such context there is a primary ffmpeg source, an "in between" stage, and a secondary ffmpeg source as result, containing the custom processed sound. This is an elegant scheme indeed. As soon as XBMC knows that there is an "in between" stage selected, XBMC is not going to sink the primary audio stream, but instead XBMC is going to sink the secondary audio stream. This way XBMC merges the custom processed audio with the video, on HDMI. Right?

Looks perfect, indeed!
Reply

Logout Mark Read Team Forum Stats Members Help
diyAudio DSP plugin0