aliasing is heared with playback 44.1kHz files on 48kHz system sample rate

[nutela]

The sound card used is an Echo Mia MIDI 4xio 24/96 sound card. When I set the sample rate in the echo.settings file to 44100 the aliasing is gone. Therefore this might be in the wrong category, please let my know and I'll change that.

[korli]

This aliasing should be related to the resampling implemented in mixer media addon.

[pulkomandy]

May be useful : ​http://radio.feld.cvut.cz/AES/atp2002/proc/paper07.pdf

[pulkomandy]

I did some tests. Here's how it goes :

• The mixer media add-on doesn't handle input format changes and assumes all the inputs are at the same framerate as the output
• As a result, no resampling at all is done : the resampler gets two buffers of the same size and just copies data between them
• Dropping or inserting samples must be done somewhere else, no idea where.

To fix this properly we have to implement InputFormatChanged in the MixerCore and tell the resampler which frame rate it should be using for the input. OutputFormatChanged should also tell it the output format, so the resampler can compute the resampling factor from the exact sample rate, rather than the buffers size.

This means the resampler should be allowed to either not fill the dest buffer, or not empty the source one completely, depending on the available data and space, and the sample rate ratio between input and output.

A lot of refactoring is needed to get all this working.

[korli]

Replying to pulkomandy:

I did some tests. Here's how it goes :

• The mixer media add-on doesn't handle input format changes and assumes all the inputs are at the same framerate as the output

I don't see how this is connected to resampling (upsampling and downsampling work IMO). Input format changes can be not handled, true. But the input format is set anyway at the connection (just check the MixerInput constructor).

[pulkomandy]

I did some tests, printing the buffer sizes in tne Resample function. No matter what sample rate is selected, the resampler always is asked to fill a 2048 frame buffer from a 2048 frame buffer. This leads to using the "no-op" resampler optimization. I have no idea how it ends up converting from 44100 to 48000 Hz that way, but it's definitely not the resampler doing its work.

Anyway, the resampler does not know about the actual framerate and try to guess it from the size of the buffers given to it, which is approximate and actually does not work at all, as both input and output buffers are 2048 samples long. You can see that in MixerCore.cpp MixerThread function :

• The call to fResampler[i]->Resample use fMixBufferFrameCount as the frame count for the source while it is calculated from the output information, and is equal to the frames_per_buffer(...) used for the destination
• Compiling the mixer add on with debug level 4 will make the PRINT call just below that show the buffer sizes are identical
• The resampling ratio computed in Resample.cpp is the ratio between these frame counts, and thus is always 1, which means no resampling is ever done.

[korli]

My test procedure: Audio setup: output 192kHz 24bits. Test file: 44,1kHz 16bits. Media player is the client application. The mixer internally converts everything to float. The media player already converts to float. Added a debug output in Resampler::float_to_float(). Resampler::float_to_float() delta:0.229687

I think the mixer is definitely upsampling. So I'm wondering what your tests actually are.

[pulkomandy]

My test setup is :

• media player playing an mp3 file at 44100Hz, 32bit float
• Sound card output set to 24bit int, 48000Hz

I'll check again.

[pulkomandy]

Ok, I found what I was doing wring. There are actually two mixer instances, one doing the sample rate conversion and anoter one converting the format. This is suboptimal, but ok, it can work.

Now, here is what I see :

• The media kit size the resampling buffer to be 1/50th of a second, that is, 960 samples at 44800Hz.
• It seems that sometime we get buffer of different sizes, but that may be caused by the debug output making things too slow. I saw some calls to resample with as low as 17 samples. In this case the computed delta is 1.13 instead of 1.08, so the interpolation isn't stable.

I fail to see the purpose of having two resamplers. But more importantly, I need to know the sample rate of input and output when constructing a filtering resampler, to precompute some tables and make the filter lighter in cpu use (and thus suitable for ral time resampling). With the way mixer nodes are instanciated, this is not possible. I can't know the input sample rate when the mixer is created, and have to guess it when I start receiving samples. What if an application decides to send longer frames (like 1second of audio at a time) ?

[stargatefan]

I don't understand why you would upsample ????? Honestly there is nothing to be gained by doing so. It would always be best to send the file in the format is was constructed in. It will also remove alot of olympics from the backend of the audio engine that simply don't need doing.

At that point just have the audio hosting application select the appropriate output sample rate. IE if the file is 44.1 have is use 44.1, if its 192khz have it use 192khz. the need to upconvert seems like such a wasted effort as the resolution of the file is still 44.1 and you going to add alot of audio artifacts to gain the extra resolution.

[pulkomandy]

You may have multiple streams at different rates playing at the same time, so the resampler is needed. It would be nice to have some 'auto' setting for the soundcard output rate, but you may as well need a fixed one (spdif for example).

So we can improve the situation, but we should fix the mixer anyway.

[stippi]

I already fixed the mixer, you may have missed the commit. I even implemented changing the resampling method on the fly. Unfortunately, your linear resampling algorithm doesn't quite work and produces audible clicks or rather very short gaps between buffers.

[stargatefan]

Replying to pulkomandy:

You may have multiple streams at different rates playing at the same time, so the resampler is needed. It would be nice to have some 'auto' setting for the soundcard output rate, but you may as well need a fixed one (spdif for example).

So we can improve the situation, but we should fix the mixer anyway.

that may very well be true but it would be in some very rare circumstances and generally would be handeled by the host application. typically what is more likely to occur is truncation and decimation to a lower sample rate IE 48khz to 44.1 as 100% of audio gear supports 16b 44.1khz.

now where things get tricky in this situation is when you would be mixing audio streams in a Digital workstation program. but those renders should be handled by the host application not the media server as latency correction acros multiple tracks could lead to some very big problems later down the road. What should be allowable is a master bus output from those applications as well as application control over input sources.

I would never want the Os to handle audio routing or encoding of the digital media for a rendered wav file in a audio application. If anything I want the Os out of the way as much as possiable.. It just flatly doesn't make sense and no audio software I am aware of does so.

[stargatefan]

Replying to stippi:

I already fixed the mixer, you may have missed the commit. I even implemented changing the resampling method on the fly. Unfortunately, your linear resampling algorithm doesn't quite work and produces audible clicks or rather very short gaps between buffers.

thats likely fixable with a accumulating underun buffer. say set if for 10,000 samples which should be a barely noticeable delay but should keep the cpu and output well ahead of the issue.

[pulkomandy]

Stippi : yes, my algorithm is 'broken'. The problem is there is no passing of values from a buffer to the next one.

• The resampling factor is recomputed for each buffer that comes in (and they may have different sizes)
• The latest value and float sample-position from the previous computation are not kept, so it is not possible to compute the first (or last) sample of each buffer properly

It's nothing impossible to solve, but it needs some thinking and makes the code slightly more complicated.

[stippi]

To stargatefan: The system works just fine in principle. Audio applications *can* do their own mixing and can have a single master output, if they so chose. Clockwerk and I think any other current BeOS/Haiku audio app is using such a setup today. There is nothing wrong with, and it's in fact required to have a global system mixer. And since sound cards have mostly one sound input, it's only logical to drive it in the highest frequency that the hardware allows (which is what Haiku currently does). Most movies use 48 kHz, most music is 44.1 kHz -- so what should one pick? In theory, the mixer could be improved to switch the audio hardware to the highest sampling rate of current mixer inputs. If there is only one input, use the sampling rate of that one. I don't think, however, that with a proper resampling implementation, it would make any practical difference. In fact it would just make the code more complicated and probably introduce audible hickups when there is already one mixer input playing and another one is added which causes a switch to a higher sampling rate.

Really, fixing the resampling is all that needs to be done here.

To pulkomandy: The Resamplers are already C++ objects which can hold arbitrary members. The same resampler object is used for a given channel/stream, so why not just implement what you propose? However I don't think that the audible artifacts are in fact caused by the problem you describe. It's just too obvious. I fully understand the problem, I just believe the effects should be more subtle. Something else is probably going on in addition to what you describe.

[stargatefan]

Replying to stippi:

To stargatefan: The system works just fine in principle. Audio applications *can* do their own mixing and can have a single master output, if they so chose. Clockwerk and I think any other current BeOS/Haiku audio app is using such a setup today. There is nothing wrong with, and it's in fact required to have a global system mixer. And since sound cards have mostly one sound input, it's only logical to drive it in the highest frequency that the hardware allows (which is what Haiku currently does). Most movies use 48 kHz, most music is 44.1 kHz -- so what should one pick? In theory, the mixer could be improved to switch the audio hardware to the highest sampling rate of current mixer inputs. If there is only one input, use the sampling rate of that one. I don't think, however, that with a proper resampling implementation, it would make any practical difference. In fact it would just make the code more complicated and probably introduce audible hickups when there is already one mixer input playing and another one is added which causes a switch to a higher sampling rate.

Really, fixing the resampling is all that needs to be done here.

Well in certain case resampling is the only option as hardware may only support one type of input and files are in another. no argument there. My concern is that mixer is really becoming more then it needs to be. whiles it great the I can watch 6 videos and listen to dozens of mp3's and or wav files all at the same time. Its not a very practical design for the end user.

Try listening to 2 different songs as the same time. Its just not something the human brain would want to do.

the ASIO model of mixer handling I think we would be a big improvement in many ways. alot of it is already in the mixer now for the most part. The overall design of ASIO looks to be based in some part on the Be mixer, which was great in its day.

My thought is that to some degree mixer might be getting to complex. the application should have the job of file conversion or the mixer master output. pick a resolution " of one thats available in the hardware and end user selectable" and just go to that regardless of what comming in.

but applications need to be able to acess that hardware " or have acess to it" down the road if DAW "digital audio workstations" are ever to take root in this OS. I sure hope they do. Would be a huge userbase gain for haiku.

so your down to the WDM and ASIO rivalry to some degree when the internal API dictates a specific behavioor that the application developer may not want, nor the end user.

BTW thats not a knock against the OS. It works great as it is now.

My thought it. Pass the selection of sample rate to the application. this leaves the API open to DAW developers. also putting in a bootstrap type of mode for audio hardware control and IO blocking. Highly recomened.

If up sampling needs doing. put it on the application or create a api handle for it. if x=y then do z, if x=f then do b.

there are lots of times where I change output resolutions in DAWS to see how things are sound at various bit depths, 32,24 and 16 bit and different bit rates.

So the DAW needs to be able to control these things.

Although human hearing and perception of audio quality over 16b 44.1 is largely pyschoaccoustic according to most testing.

but there are benefit to mixing a song at 24b 192khz and then rendering at 44.1 16b. Mostly more dynamic headroom and less output congestion.

[bonefish]

Replying to stargatefan:

Well in certain case resampling is the only option as hardware may only support one type of input and files are in another. no argument there. My concern is that mixer is really becoming more then it needs to be. whiles it great the I can watch 6 videos and listen to dozens of mp3's and or wav files all at the same time. Its not a very practical design for the end user.

Try listening to 2 different songs as the same time. Its just not something the human brain would want to do.

If you listen to music and another application (chat for instance) plays a notification sound, some component has to mix the two. And that's the mixer's job. I suspect that there are workflows where audio people use more than one sound generating application at a time. Again, the mixer's job.

there are lots of times where I change output resolutions in DAWS to see how things are sound at various bit depths, 32,24 and 16 bit and different bit rates.

So the DAW needs to be able to control these things.

Good that all of this is possible already. If an application just wants to control the hardware node's parameters, it can use the respective parameter web -- that's exactly what the media preferences application is doing. If the application wants full control over the output, it can disconnect the mixer and directly connect to the hardware's node (this would, of course, shut up all other applications). So I don't see a show stopper for whatever kind of audio application you have in mind.

At any rate, if you want to discuss the design of Haiku's media framework, please use one of the mailing lists.

[stargatefan]

Replying to bonefish:

If you listen to music and another application (chat for instance) plays a notification sound, some component has to mix the two. And that's the mixer's job. I suspect that there are workflows where audio people use more than one sound generating application at a time. Again, the mixer's job.

Well usually in a audio application the sound gerneration is handled in the hosting or mixing program itself. IE the program write all the rendering to the master output and typicaloly handles all associated sound that need mixing as well.

I hadn't thought about beeps and buzzers for notifications. typically I turn those things off. Good reminder !

Good that all of this is possible already. If an application just wants to control the hardware node's parameters, it can use the respective parameter web -- that's exactly what the media preferences application is doing. If the application wants full control over the output, it can disconnect the mixer and directly connect to the hardware's node (this would, of course, shut up all other applications). So I don't see a show stopper for whatever kind of audio application you have in mind.

It good to know. I am looking at trying to build a application around the bepia for haiku but built of the ideas and code of other DAW's. Lots to learn here. Just wanted to make sure that whatever fix got implement for this issue, did not block those applications later requireing a massive amount of work to unfix it.

[stargatefan]

I have tested hundred of file formats at various bit rates. No alliasing heard anywhere with any of them. The only exception to this is gnash crashing the media server mixer thread but thats not part of this ticket. I think it might be well beyond time to close this one and get it off the bug list.

[stippi]

This bug should stay open, it has not been fixed. Bilinear resampling has been implemented, but some problems remain with it and it's not activated by default. If you hear the aliasing effects depends on your hardware.

[pulkomandy]

I can't hear any problem with the linear resampling (it's not bilinear, btw, only linear). It seems to work fine as is for me.

[pulkomandy]

Well, works for me, so reassigning to someone that can reproduce the bug ?

[pulkomandy]

Oh, I noticed the code for downsampling is broken. Stippi, is this what you are using ? May explain why I didn't notice it, I'm using the upsampler way more often.

[pulkomandy]

Ok, the interpolating resampler is now fixed for good in hrev42435. (commit message includes a way to test it).