[steph_tsf]

Thanks to the {primary ffmpeg source / in between module / secondary ffmpeg source} architecture, a same diyAudio DSP plugin could run on Windows, Mac and Android. How so ?
Let's code the "in between" module in C language, implementing a stereo 3-way crossover obeying the ffmpeg standard regarding Sink and Source. Such code would compile using a x86 compiler for running on Windows 8, and would compile using a ARM compiler for running on Android. Right?

[negge]

Found this thread by accident and I'd like to chip in with a few thoughts.

I like the approach that FernetMenta has been advocating - an API where addons can hook into the audio processing stage to do any kind of processing. At first it sounds like something totally over the top that only audiophiles would use, but it could have more mainstream purposes as well, for example

- standard equalizer. Everyone knows how to use an equalizer (whether people actually should use one is another question)

- pseudo-surround upmixing could be done in software. To make it more interesting a hypothetical addon for this could be able to query XBMC for the type of media being played (music/movie) and that way determine if something like DTS Neo6 should be used or if something like Pro-Logic would be more appropriate.

- "midnight mode" (this should really be a core feature though)

- "port replicator" addons - basically something that could take whatever audio XBMC is outputting and return it untouched plus copy it to another audio output. Would atleast please the people over at the dual audio output thread. This way XBMC could be configured to use passthrough and HDMI and all the fanciness while the addon could take care of outputting a stereo downmix through the analog output.

I'm just brainstorming here, some of the ideas may not even be technically possible. Midnight mode is the only one I would actually use.

[steph_tsf]

XBMC is not what you think. XBMC needs to be seen as a presentation layer coming on top of ffmpeg. Thus, currently, in case you want XBMC to PROCESS the audio the way you want, XBMC needs to ask to ffmpeg. In case ffmpeg doesn't support what you want, you "violate the XBMC architecture". In case you *really* want XBMC to PROCESS the audio, you need to comply with complicated standards, still evolving. You will lose a lot of time. There will be a lot of regression bugs.

At the end of the day you will find yourself installing JACK. JACK is a pseudo realtime Server dedicated to Audio, more than a plain simple JACK patch panel. You will try grabbing the audio out of XBMC (I should say: out of ffmpeg), asking JACK to pick-up that audio signal, then ask JACK to route it to some audio DSP that you wrote, and finally ask JACK to take care of the physical interface, like sending audio through HDMI using the HDMI multichannel PCM modality and associated software driver.

At the end of such process, you'll want to design your own audio player, natively supporting everything you want. While doing so, you would jump on the Android bandwagon, delivering one source code, compiling it once (for Android ARM), and posting it on Google Play. Possibly asking a few dollars for it.

Within 12 months, all Android Tablets will be equipped with quadcore or octacore ARM processors. Anybody targeting professional audio quality, requiring deterministic reaction times and a latency as low as possible, will continue relying on dedicated audio servers like JACK is nowadays. With Android, the big difference will be that on Android Tablets, one or two Gigahertz ARM cores can be dedicated to Audio. One ARM core will do the resampling, especially when there is the need to mix many audio sources exhibiting different sampling frequencies. One other ARM core will do the Audio DSP like equalizing, dynamics compression, speaker crossover, etc. When there are very few unrelated channels to mix, one dedicated core can do both resampling and DSP. This time, there will be a global audio quality improvement, as the audio will get processed in real realtime, not anymore in pseudo realtime. This time, the Audio Server (call it Gack if you want) gains full control of the Gigahertz core it is using. One interesting consequence is that for Linux, there is almost no audio workload anymore. Consequently, the GUI can operate faster, scanning the control surfaces faster, and updating the graphics faster. This is the benefit of a dedicated, deterministic multiprocessing scheme. Linux GUI and Audio DSP don't conflict anymore.

Basically, you had this in computers like NeXT, Atari Falcon, SGI Indigo workstations, 20 years ago. Unfortunately, Motorola charged a lot of money for the DSP56K chip. This prevented such solution to become mainstream. The situation has now changed, with Mediatek producing the MTK6582 (quadcore ARM) and MTK6592 (octacore ARM) chips, at very affordable prices.

Google the iNew i6000 MTK6592 Octacore. Visualize this, becoming your next DAW. Natively supporting 32-bit audio. Supporting also 64-bit audio.
Yes, we can (provided we dump what needs to be dumped).

[dzuvela]

hi steph_tsf
have you seen this: http://rtaylor.sites.tru.ca/2013/06/25/d...are-howto/

link and author is mentioned no more no less then here: http://www.linkwitzlab.com/LX521/DSP_challenge.htm
scroll down to:
"b) Digital Crossover/EQ with Open-Source Software: HOWTO"

key-components are: "ecasound", "MDP", "SOX", and of course alsa, oss, jack,...virtualy anything..... and finally magic word: pipelining

I belive that with components above is possible to redesign xbmc audio architecture in to DSP.
But as few developers mentioned here, this would make xbmc unusable for average XBMC user.
Some "how to" would be better way to make agile diyAudifiles happy.

[AchimTuran]

Very long text wall....

When I read through the thread here, I had the idea of using python for audioprocessing. Then we are plattform independet. What do you think about it?

Is there anyone wokring on such audio API/Plugin?

If yes I would like to help!

[FernetMenta]

Currently no one of team XBMC because we are short final for Gotham release.

[AchimTuran]

Yes, I have read and fritsch also already told me this.

But what do you think about python as addon / plugin for audio signal processing? We can call python scripts from the ActiveAE C++ code (Call python from C sorry only found it in german)

At the moment I try to implement an convolver in ActiveAE (ActiveAE convolver) If the first implementation is finished, I might in the second phase replace the convolver as a Python script.

It should not come over so that I XBMC team wanted to do the implementation, but that I wanted to do something.

[FernetMenta]

I already explained in post ~10 in this thread. First an extension point in AE should be defined and created. Then addons can register and use this extension point for additional processing.
Feel free to get started on this. Contribution is always appreciated.

[RayGarrison]

The Audiophiles who want DSP are an extremely small minority, probably less than 1% of our userbase, thus I don't want to see anything that might confuse the user who couldn't care less about DSP. This is something that needs to be done outside of the current settings, maybe a new screen or preferably via an Add-On in my view.

I'm completely new to this forum, and quite new to XBMC, so I feel somewhat presumptuous posting anything in this thread. I thought my reaction might be helpful. I really disagree with the above thought. I'm an "audiophile", in the sense that I really enjoy listening to music, and for the past 40 years or so I've been buying music I like and hardware that makes me happy when I'm listening to that music. I've migrated to digital media, computers as music servers, and media players instead of turntables / CD players because I have access to more music that way, and I can do things to make it sound better in my home than I could do without computers. I have a lot of friends who are "audiophiles" and they've pretty much done the same thing. DSP is one of the really cool things that computers offer us that we couldn't really do well before computers. Parametric equalizers to smooth bass response, flexible crossovers (frequency / phase) to hook up a subwoofer without messing up the sound, cross feed processing so headphone listening sounds more natural... these, and many other cool gizmos, are one of the reasons why I and so many of my friends are using stuff like Foobar when we listen to music. (Or I was until I found XBMC.) I'd say that the number of "Audiophiles who want DSP" is closer to 100% than it is to 1%. Whether DSP is something that belongs in XBMC, is available as a plugin, or is totally out of scope for what XBMC is are all great discussions. I just hope you don't presume during those discussions that the market you're targeting is indifferent, or worse yet opposed to, DSP as part of the music experience. We're (at least the part of "we" that I belong to) are not.

[fritsch]

What he said was: That we have some millions audiophiles that want something and no one of them is stepping up to actually do something.

[NotMediaSavvy]

Sorry fore the "long ish" post, but I have not posted yet and have a few things to say..

As someone who makes their living from selling plugins for the recording industry, I have some thoughts I'd like to share.

I had a teacher in school who taught DSP (among other things) and he gave me the greatest advice I ever got. I know you all have heard this too, but from what I see, you may not have implemented it. He would always say "K.I.S.S.", as in Keep It Simple Stupid. Yeah, I used to get that a lot from him because I would end up writing a whole series of steps when a simple OK button would do.. So, lets look at this as if we're stupid!

Android and iOS
Ok so, if you are watching a movie on one of these devices, do you need audio DSP? probably not.. You are watching it on a tiny screen to begin with so it hardly calls for HUGE speakers and sound system..

Just implement LADSPA for Linux, VST for Windows, and AU for Mac.. These are all proven standards that we (dsp developers) know like the back of our hands, the only one that in the past seemed to be flaky was LADSPA and I think that was more OS issue than actual LADSPA technology issue. VST and AU both have a proven track record, free licensing, and literally hundreds of thousands of freeware plugins available right now. If you want to make XBMC as popular as it can be then implement these technologies..

I for one do not plan on learning a new interface that's used for only a single program (XBMC). Now if it were a case that one of my existing plugs needed a simple edit to make it more useful for XBMC then fine I'd edit it, but I am not about to change the way in which it receives its data stream.

HTPC no matter how popular we think it is, in reality it's still somewhat of a niche. So, make it easy for us to support you and we will, of that you can be sure. You'll have more DSP plugs than you can shake a stick at! Overnight!

Thanks for taking the time to read that, and that will be the last wall of text you can expect from me..

---

What he said was: That we have some millions audiophiles that want something and no one of them is stepping up to actually do something.

Implement VST, AU, and LADSPA and you'll see how fast we step up. Don't make us learn some new unproven interface technology that you've designed because we just won't. These technologies are proven and are the standards in use today! Do you know how many companies will instantly be able to supply XBMC users with already made and proven plugins. Thousands of companies exist right now, and hundreds of thousands of plugins exist..

[jmarshall]

It's not about the plugins. It's about someone that cares implementing an interface (any interface) in XBMC, regardless of what that interface might be.

[fritsch]

Implement VST, AU, and LADSPA and you'll see how fast we step up. Don't make us learn some new unproven interface technology that you've designed because we just won't. These technologies are proven and are the standards in use today! Do you know how many companies will instantly be able to supply XBMC users with already made and proven plugins. Thousands of companies exist right now, and hundreds of thousands of plugins exist..

You exactly made my point. So step up and implement it - you want this feature, you do it, that was the whole point. And I don't care a single bit about a company out there, that might sell more plugins cause I implement that widespreed interface their plugin works for in my freetime.

[FernetMenta]

XBMC's audio engine is platform independent and you can't just make use of platform specific APIs like LADSPA at that layer. What's needed to be done first is to implement an abstraction layer between the audio engine and whatever plugin devs prefer to interface with their plugins.
XBMC has an established addon concept which can be used here. There is no need in making the architecture decay by a misinterpreted keep-it-simple approach.

[RayGarrison]

That sounds perfectly sane. I'm not exactly the sharpest knife in the drawer when it comes to coding, being more of a system architecture and business-to-tech-guy translator, but if you can point me at the best place to start looking at the source components of the audio engine where this abstraction layer would be implemented as an addon I'll be happy to see whether it's over my head or not. I know the whole project is available on the XBMC Github repo, but it would be helpful if I had a starting point to start reading. Is this kind of addon most likely to be done in C++ or Python, or is it agnostic, or are there any other options, or am I wasting time by asking these questions when I should just open the repo and start reading ?