KDE2, Multimedia and aRts
What it this all about?
A real desktop wouldn't be complete for most people if it didn't provide any multimedia capabilities. Definitely, a lot of people want to listen to mp3 or midi files, want to have plings when their windows close, and occasionally, they want video, tv, mod, xm, recording, game sounds, etc.
The applications are broad and also include things like voice recognition, internet telephony, multimedia editing and publishing or simple things like tetris.
KDE1 did try to provide very simple multimedia capabilities to applications, to make it easy for instance to make your windowmanager to do the pling. And some applications, like kreversi or kwmsound, successfully used this stuff.
But usually, everybody doing multimedia under KDE1 did reinvent the wheel. Writing to the soundcard, reading file formats, writing an api for filters, doing midi processing one way or the other, etc.
You may say: "Well, I don't mind, it somehow works!" But in fact, you will see the limits easily. When you for instance play an mp3 with some mp3 player, your windowmanager pling will disappear.
When you want to play midi files, you get two or three different applications (kmid, kmidi and kmedia), which somehow perform the same tasks, but differently. Well, but no need to bother you with more examples. What can we change?
Let me introduce aRts. It's an ancronym for "analog realtime synthesizer". On the KDE-TWO developer conference, we decided that aRts will - in some way or the other - be a part of KDE2. The relevant part of aRts is that it is a sophisticated a multimedia component system. Small modules, which do simple tasks like adding two signals, or high-pass-filtering one signal can be arranged in flow graphs.
So the idea is, that programmers only need to write simple components, while you can compose them in an infinite number of combinations to new things.
This is useful for "real" music applications, like sequencing software, synthesis and building virtual studios.
But on the other hand, you can also use that technology to handle all the other cases that are required in an desktop environment, like playing mp3s, midi files, serving voice recognition while to watching your TV program, serving window manager sounds while playing mp3s, serving games while still being able to hear your end-of-download sound.
That was the primary idea behind choosing aRts for KDE2. Having an esd-deluxe, that can't only mix something together, but intelligently run just those components you need, in the background. If you want to think in terms of other products which are related, think of the multimedia APIs that for instance BeOS provides, or the major audio plugin technologies on Windows.
On the other hand, think of the analogy: I have one graphic card, and a lot of programs use it in a sensible way of communication and interoperation, since I have an X11 server. So aRts will be the X11 server for audio. Not just mixing, but: sensible way of communication and interoperation.
Still, there are open questions remaining, which probably will only be decided while the integration work is actually done. One is for instance CORBA. aRts currently uses CORBA to handle component interaction, remote access, etc. But for realtime this is not the optimal solution, as CORBA adds unneccessary overhead. It also simply wasn't built to handle real traffic. So there might be the possibility of creating a new, highspeed IPC layer, which will just be used for multimedia communication. But don't confuse this with DCOP here, I said: a new IPC layer.
These are things that will show in the next months. Also the task of solving every multimedia task and protocol elegantly is probably too much for KDE2, but well, there will be further KDE releases later.
We decided to start with audio, at least. So kaudioserver should be replaced by aRts, which is more modern, component based and more flexible. Client APIs need to be provided, to make playing a wave file a trivial thing to do.
Most people that have seen a live demonstration of aRts at KDE-TWO were impressed. Basically, things like playing mp3's through aRts component system, while listening to window manager sounds just work. Well, this would be nothing special, if the aRts component system couldn't handle putting reverb on your mp3, or on your window manager sounds. Or simultaneously recording things. Or doing midi synthesis. Or playing two mp3s and mixing them over a virtual mixer.
So what this shows is: the technology works, is flexible and powerful. And that needs to be integrated properly now. So that is what we're going to do for the multimedia stuff in KDE2.
In fact, there may be other places where it could pop up. While the KDE2 commitment shows that aRts has a lot of potential, aRts doesn't need not be restricted to KDE. No part of the aRts infrastructure will require Qt or KDElibs, and other people are free to adopt aRts technology to whatever they need it for. Especially, if for instance Gnome wants to work with aRts in the multimedia department, they are welcome. The possibility to throw CORBA out, and replace it with a self contained IPC mechanism, might even improve the situation here, as then aRts would depend only on itself, and not on mico or other CORBA technology.
Finally: if you want to help in the development, please join the kde-multimedia list. If you want to play with current (KDE1 based) aRts releases or find out more about aRts, see http://www.arts-project.org/. If you want to see the integration in KDE2 - either use CVS - first things should pop up there pretty soon - or stay tuned until KDE2 is out.
Basically, I don't like announcing things before they are completely done, as I am mainly a programmer. I am interested in getting things done. But on the other hand, lot's of questions have been popping up, and people seem to be uncertain of what is happening in the multimedia department. So please, view this as orientation, not as advertising.