Spare HDA jacks available - how can we address them?

[dsuden]

I don't know if it's already possible somehow, but if there is some way to make the multiple output jacks on HDA-equipped motherboards software-selectable, there are all kinds of exciting possibilities for both multi-track output and discreet selection by individual software programs.

[dsuden]

We're testing motherboard-based HDA audio, and it lists a variety of outputs in Media Prefs, corresponding to multiple output jacks on the computer.

Media Prefs listings...

In Audio Settings, I have a choice for Audio Output of "HD Audio." For channel, I have a choice of "outout 0" and that's it...BUT...

Under "HD Audio, in the Output tab," I see these options listed:

Front Rear Center/Sub Side

In that part of Media Prefs, it's possible to mute and unmute, but not select them.

Interesting to note that main channel Haiku audio is feeding the same audio out of all the outputs right now. Presumably, as I've read about HDA, each of these output jacks could be made available as separate audio channels, addressable by software...thus multi-track audio output would be possible, as well as allowing individual audio programs to send output to the outputs of their choice.

If I do listdevs I get the following information on this specific hardware:

device Multimedia controller (Audio device) [4|3|0]

vendor 1002: Advanced Micro Devices [AMD] nee ATI device 4383: SBx00 Azalia (Intel HDA)

device Multimedia controller (Audio device) [4|3|0]

vendor 1002: Advanced Micro Devices [AMD] nee ATI device aa90: Turks HDMI Audio [Radeon HD 6000 Series]

So, I'm trying to determine what would be required, inside the operating system code, or via coding in a third party program like one of ours, to get at those additional outputs and make use of them.

Inviting LOTS of feedback on this. We would REALLY like to find a way to make this work, and any efforts on our part would happily be donated back into the Haiku community.

[SeanCollins]

I don't have much to add except that a api similar to asio would be smart to implement. It should be completely feasible to route inputs and outputs via some api mechanism to and from audio/media applications.

I think Asio "if a external api were to be adopted" would be the Best choice to use as its widely adopted and well documented and a proven design.though asio could be a media add-on exsposing the lower levels of the mixer driver interface which might be preferable to the system mixer ?????

A native api obviously is prefferable likely easier to implement. I don't think the short coming is the drivers as much as the system mixer not having the capability currently. I know that work was done in ice1712 to allow multi input output similar to the ticket description. I don't see a way to do so however at the mixer level. Which imho is the best place to do this. Allow the mixer to push available inputs and outputs to applications allowing for easy syncronization, vrs adding a alien low level frame work that add more software complexity.

Of course, some of this could be ignorance on our parts as maybe the functionality exists but is poorly documented. Someone should Ping Pete Goodeve and Stippi they know bowels of the media system and would be fiarly helpful. We should also put a mail on the development list so this ticket doesn't get crowded.

[dsuden]

Sean, if you are on the dev list, is this something you could start the ball rolling on, since you understand better than I what's involved with it?

[korli]

Any application can already connect directly to the multiaudio node inputs/outputs. In fact exposing available outputs and inputs in HDA is still to be implemented.

FWIW ASIO API equivalent is more or less Multiaudio API. Any application can bypass the Media Kit completely and talks directly the driver via the Multiaudio API.

[SeanCollins]

@Korli

Could you please point me in the direction of the Multiaudio api doc or code, I looked through a ton of documentation but it looks like it is something that was added post Beosr5 since it appeared to be unfinished in Beosr5. Either that or I just missed it completely.

@dsuden

I'd like to know more about what it is specifically we would be asking about. What tools and features exist now and will be needed in the future. But yes I have a first draft of the mail for the dev mailing list almost ready.

[dsuden]

To some degree, I think the available audio hardware may ALREADY be published, since in our software, it is possible to select between two different sound cards as output sources. So maybe what we're really talking about is an issue of an unfinished HDA driver, as Korli mentioned. If it's the latter, maybe we could set up a bounty? We'd certainly contribute a good amount toward that since we really need it, and fairly soon.

[ttcoder]

... or if 'talking directly to the driver' (as mentionned by korli, thx!) is not too much of a hair-puller maybe I can take a stab at that... This would make sense especially if implementing the Media-Kit/multiaudio <-> driver glue is enough work that it won't happen anytime soon...

@korli I wonder how to talk to the driver: presumably it exposes 'ports' to the userland world, so in my application I would call find_port() on a hardcoded name (?), then talk it into selecting the specific "node" (for a given audio jack of my choosing) and send audio data to that same port ? In truth I'm much tempted to let Sean & Dane organize a bounty ..etc to allow you to finish that aspect of the driver.. Though then I would still have to figure out the multi-audio API of course (it's not at api.haiku-os.org)... If you're sure I can dive into the header files (which ones, the HDA driver's, not the Media Kit's, right?) and have something to show for it when Dane asks, I'm all for it ;-)

[to recall my non-userland track record: my "skills" are limited to having fixed a bug in the HDA driver for my R61 thinkpad, and even that "bugfix" was done with brute force tracing, not actual understanding of the driver or the media kit internals]

[ttcoder]

With some search-engine help I've found this ​multi_audio_test.cpp sample file. It works with ioctl()s on the /dev/audio/hda..... pseudo-file. Great, I thought! But no cigar :-/.. Here's my findings..

~/Desktop/haiku_GIT/MULTIAUDIO_test> ./multi_audio_test /dev/audio/hmulti/hda/0 
multi_audio_test: Could not open "/dev/audio/hmulti/hda/0": Device/File/Resource busy

Ok so the media_server needs to be shutdown before accessing the driver.. Is that a bug, i.e. something to report in this ticket (or another one) ? If that's a "feature" then it's definitely a show-stopper for us...

Then I did...

~/Desktop/haiku_GIT/MULTIAUDIO_test> hey media_server quit
(....)
~/Desktop/haiku_GIT/MULTIAUDIO_test> ./multi_audio_test /dev/audio/hmulti/hda/0 
> desc
friendly name:                  HD Audio
vendor:                         Haiku

output rates:                   0x2900 (max 0)
input rates:                    0x900 (max 0)
max cont. var. sample rate:     0
min cont. var. sample rate:     0
output formats:                 0x150
input formats:                  0x150
lock sources:                   0x1
timecode sources:               0x0
interface flags:                0x3
control panel string:                   

channels:
  2 outputs, 2 inputs
  2 out busses, 2 in busses

  ID    kind       designations connectors
   0    out        0x2000001    0x0
   1    out        0x2000002    0x0
   2    in         0x2000001    0x0
   3    in         0x2000002    0x0
> channels
4 channels:
  xxxx
> play
format 32bit (0x1000)
sample rate 48000 (0x100)
playback: buffer count 2, channels 2, buffer size 2048
record: buffer count 2, channels 2, buffer size 2048

but did not hear anything coming from the speakers...

(yep audio works fine otherwise on this laptop: I'm listening to music right now, after restarting media_server).

Will leave interpretation to somebody knowledgeable..! Much appreciated in advance thanks :-)

[korli]

multi_audio_test.cpp requires exclusive access to the device. It only copies recorded buffer to playback ones: what you hear should be what you record. This can be used for measuring the audio latency for instance.

[ttcoder]

Here's a snapshot, for the sake of keeping an online summary of off-ticket discussions:

• a) we clearly want to keep the driver talking to media_server as normal, and access multi-audio through the media_server only.
• b) turns out the userland code to access multiple streams probably won't be hard: the e.g. tunetracker "File Audition" code (released under a MIT/BSD license for other apps to use) which implements targetting a specific audio card (other than the "main" one) might be used for accessing a specific audio jack (other than the "line out" one). (<The multiaudio addon will expose an input for each stream exposed by the driver. The user can select this input in the Media preflet for the mixer output.>)
• c) the key to solving this ticket, thus, is to implement code driver-side; so that the HDA driver 1) allows for more than just one output stereo stream, 2) can route that output playback stream to an output "widget" in HDA parlance and then 3) route that output widget to an output connector (a jack other than the main output jack).

So it seems out of the above 3 points, #b is something I can handle on my own and only for #c do we need help. As soon as I have time I'll get back to reading/understanding the HDA code and see if I can contribute more than moral encouragement for the "multiple playback streams" ..etc stuff ;-)

[ttcoder]

In a few days I'll try the "2 streams with 2 channels each" approach. Today I tried the "1x4" (1 output stream with 4 channels) approach but it does not seem to work:

• Modified hda_multi_audio.cpp so that get_description() returns 4 outChannels of type B_MULTI_OUTPUT_CHANNEL and so that get_enabled_channels() enables them.
• Toyed with hda_controller.cpp (see below).

Things looked good initially: the media preflet drop-down box does list 2 items instead of one now (output 0 and output 1).

I then tried to select "output 1" (instead of 0) in the media Preflet and restarted media a couple times, but apparently there is a bug in the media_server: when running Cortex I see that the main Mixer is still connected to HDA's first input, not to the second. Or maybe the bug is in the driver, maybe it needs to expose something else publicly so the media_server will instruct it to aim channels #2 and #3 at the output instead of channels #0 and #1... Did not find any reference to that in the source though, except for the ​hda_controller one I toyed with a bit. I can include a screenshot of Cortex if that helps, the mixer is still connected in the same way no matter which "output" is selected in the preflet.

Lastly I tried to force a connection to that second output from TTCC, and then from the "ToneProducer" addon but still to no avail. This is likely not a user error on my side, because connecting TTCC or ToneProducer works fine when I do that to the first output...

Had I been able to select one or the other that would be promising, and the next test would have been to send the compiled driver to Dane for him to try simultaneous playback of two different audio playbacks to two outputs (my HDA here only has one output)... But if the basics won't work here it won't work for him in a more complex setup either. So next I'll turn my attention to the "twin streams" tactic I think, though that will be a tougher nut to crack.

[ttcoder]

Testing the original output channel and the extra one in Cortex

[ttcoder]

Tu cut a long (very very very long :-) story short, it seems we have a success as of this afternoon. It seems (testing by sending builds to Dane through email confuses me more than if I was testing locally sometimes!) that the necessary ingredient to that recipe were to have both the "TODO" in hda_controller.cpp resolved and also add a new verb, #define VID_SET_CONVERTER_CHANNEL_COUNT 0x72d00, to the vocabulary of the driver.

/boot/common/cache/tmp/....haiku..../haiku/src/add-ons/kernel/drivers/audio/hda> git diff
diff --git a/src/add-ons/kernel/drivers/audio/hda/driver.cpp b/src/add-ons/kernel/drivers/audio/hda/driver.cpp
index e0d05f8..18982eb 100644
(..)
diff --git a/src/add-ons/kernel/drivers/audio/hda/hda_codec_defs.h b/src/add-ons/kernel/drivers/audio/hda/hda_codec_defs.h
index 1c4d021..9148c0e 100644
--- a/src/add-ons/kernel/drivers/audio/hda/hda_codec_defs.h
+++ b/src/add-ons/kernel/drivers/audio/hda/hda_codec_defs.h
@@ -69,6 +69,7 @@ enum pin_dev_type {
 #define VID_SET_CONVERTER_FORMAT                       0x20000
 #define VID_GET_CONVERTER_STREAM_CHANNEL       0xf0600
 #define VID_SET_CONVERTER_STREAM_CHANNEL       0x70600
+#define VID_SET_CONVERTER_CHANNEL_COUNT                0x72d00//
 #define VID_GET_DIGITAL_CONVERTER_CONTROL      0xf0d00
 #define VID_SET_DIGITAL_CONVERTER_CONTROL1     0x70d00
 #define VID_SET_DIGITAL_CONVERTER_CONTROL2     0x70e00
diff --git a/src/add-ons/kernel/drivers/audio/hda/hda_controller.cpp b/src/add-ons/kernel/drivers/audio/hda/hda_controller.cpp
index 350e36a..40e6c19 100644
--- a/src/add-ons/kernel/drivers/audio/hda/hda_controller.cpp
+++ b/src/add-ons/kernel/drivers/audio/hda/hda_controller.cpp
@@ -570,10 +570,10 @@ status_t
 hda_stream_setup_buffers(hda_audio_group* audioGroup, hda_stream* stream,
        const char* desc)
 {
-       uint32 response[2];
+       uint32 response[3];
        physical_entry pe;
        bdl_entry_t* bufferDescriptors;
-       corb_t verb[2];
+       corb_t verb[3];
        uint8* buffer;
        status_t rc;
 
@@ -707,6 +707,7 @@ hda_stream_setup_buffers(hda_audio_group* audioGroup, hda_stream* stream,
        dprintf("hda: stream: %ld fifo size: %d num_io_widgets: %ld\n", stream->id,
                stream->Read16(HDAC_STREAM_FIFO_SIZE), stream->num_io_widgets);
        dprintf("hda: widgets: ");
+       dprintf("\n");
 
        hda_codec* codec = audioGroup->codec;
        uint32 channelNum = 0;
@@ -720,9 +721,19 @@ hda_stream_setup_buffers(hda_audio_group* audioGroup, hda_stream* stream,
                        val = 0;
                verb[1] = MAKE_VERB(codec->addr, stream->io_widgets[i],
                        VID_SET_CONVERTER_STREAM_CHANNEL, val);
-               hda_send_verbs(codec, verb, response, 2);
+if(i%2)
+               verb[2] = MAKE_VERB(codec->addr, stream->io_widgets[i],
+                       VID_SET_CONVERTER_CHANNEL_COUNT, 3);
+else
+               verb[2] = MAKE_VERB(codec->addr, stream->io_widgets[i],
+                       VID_SET_CONVERTER_CHANNEL_COUNT, 1);
+               hda_send_verbs(codec, verb, response, 3);
                //channelNum += 2; // TODO stereo widget ? Every output gets the same stream for now
+if(i>=2)
+channelNum = 2; // TODO stereo widget ? Every output gets the same stream for now
                dprintf("%ld ", stream->io_widgets[i]);
+               dprintf("widget #%ld <= channels %ld/%ld belonging to stream #%ld (val=0x%lx) v0=%lx v1=%lx v2=%lx\n",
+                       stream->io_widgets[i], channelNum, channelNum+1, stream->id, val, verb[0], verb[1], verb[2]);
        }
        dprintf("\n");
 
diff --git a/src/add-ons/kernel/drivers/audio/hda/hda_multi_audio.cpp b/src/add-ons/kernel/drivers/audio/hda/hda_multi_audio.cpp
index 0c50f8b..e1d8cef 100644
--- a/src/add-ons/kernel/drivers/audio/hda/hda_multi_audio.cpp
+++ b/src/add-ons/kernel/drivers/audio/hda/hda_multi_audio.cpp
@@ -16,7 +16,7 @@
 #      undef TRACE
 #endif
 
-//#define TRACE_MULTI_AUDIO
+#define TRACE_MULTI_AUDIO
 #ifdef TRACE_MULTI_AUDIO
 #      define TRACE(a...) dprintf("\33[34mhda:\33[0m " a)
 #else
@@ -85,6 +85,7 @@ get_description(hda_audio_group* audioGroup, multi_description* data)
        int32 outChannels = 0;
        if (audioGroup->playback_stream != NULL)
                outChannels = 2;
+outChannels = 4;
 
        data->output_channel_count = outChannels;
        data->output_bus_channel_count = outChannels;
@@ -144,6 +145,8 @@ get_enabled_channels(hda_audio_group* audioGroup, multi_channel_enable* data)
        B_SET_CHANNEL(data->enable_bits, 1, true);
        B_SET_CHANNEL(data->enable_bits, 2, true);
        B_SET_CHANNEL(data->enable_bits, 3, true);
+       B_SET_CHANNEL(data->enable_bits, 4, true);
+       B_SET_CHANNEL(data->enable_bits, 5, true);
        data->lock_source = B_MULTI_LOCK_INTERNAL;
 
        return B_OK;

I found the new verb in the official HDA doc (thx for the pointer korli!).

Maybe that new verb could be useful to improve compatibility also (make the driver work for more people) since the doc says that verb should be sent with the other two; though its main usefulness if for us to have multiple streams to multiple jacks of course :-)

As you'll see above I've leveraged Dane's 4 output jacks to create 4 different combinations of verb/parameters; I still don't know for sure which of the four combinations is the succesful one because the jacks' physical ordering seems to differ from their programmatic ordering. Point is, Dane now has the main output on 3 jacks, and a nice "Preview" output on the remaining 4th jack :-)

Will confirm and clean this up and later we can see if this can be turned into a normal piece of code by opposed to a quick-n-dirty hack.

[korli]

I wasn't aware there were specification updates. Added some bits in hrev44592.

VID_SET_CONVERTER_CHANNEL_COUNT shouldn't be needed for mono/stereo streams.

[ttcoder]

@korli you were right; not sure how I got that confused but.. This week I started clean, did an hda driver "mod" from scratch and sure enough, it works without using VID_SET_CONVERTER_CHANNEL_COUNT at all, just as well as the old driver modification did.

Status update on this ticket.. Due to limited availability of our current "reference" motherboard we're trying to be even more flexible in terms of re-routing jacks, so that we can choose the next motherboard(s) from a wider set, even among those with only 3+2 jacks. Currently I'm mod'ing the driver to read settings at config/settings/kernel/drivers/hda.settings but that is not quite in the BeOS/Haiku philosophy so in the end we'll probably try to turn that into a GUI in the Media prefs panel maybe.

[SeanCollins]

Replying to ttcoder:

@korli you were right; not sure how I got that confused but.. This week I started clean, did an hda driver "mod" from scratch and sure enough, it works without using VID_SET_CONVERTER_CHANNEL_COUNT at all, just as well as the old driver modification did.

Status update on this ticket.. Due to limited availability of our current "reference" motherboard we're trying to be even more flexible in terms of re-routing jacks, so that we can choose the next motherboard(s) from a wider set, even among those with only 3+2 jacks. Currently I'm mod'ing the driver to read settings at config/settings/kernel/drivers/hda.settings but that is not quite in the BeOS/Haiku philosophy so in the end we'll probably try to turn that into a GUI in the Media prefs panel maybe.

Try to keep in mind that there are many audio cards which support lots of inputs and outputs, any exspansion of routing etc, really should be made available through the media server and the prefrences panel so that any and all applications future or past can make use of these changes. Also recording/capturing/using several inputs and outputs imultanoeusly should be a goal as well. Likely not hard to implement, just lacking ffrom media kit.

[ttcoder]

Just noticed tickets #6514 and #9477. The former (6514) might be already implemented in the media infrastructure used in Haiku in the last few years.

The latter is interesting and relevant to this ticket: 9477 starts from the claim/fact that flexible routing of outputs is already implemented (I don't have an Echo Layla 3G to test that claim, but things certainly work well when I hack the HDA driver to provide support ;-), and focuses on usability fine-tuning advices..

The take-away for me are..

• if the "Output channel mapping" panel of Media Preferences is the GUI we're looking for, maybe I can later get rid of the kernel/hda.settings configuration file my hacked driver is currently using. In fact, maybe those settings are already applied to my hacked HDA driver we're currently using in production ? (note to self: if said settings ever get corrupted, that would explain why we sometimes see one output get silent).
• if (as per this ticket) we bring HDA in line with multi-audio, we could follow the lead of the Echo Layla 3G driver (presumably that's /boot/system/add-ons/kernel/drivers/bin/echo3g), we could make the same decisions it made ..etc (hoping they are compatible with what we need here at TT Systems, i.e. separate outputs for "broadcast audio" and "general audio").

EDIT: looks like it took me no longer than 3 years to fully understand korli's comment:6 ;-)

EDIT2: and I still have more learning to do..

Looked at the Echo driver and now I'm not so sure that ticket #9477 relates to this one..

Maybe the "multi"-ness of the Echo card does not lie in its having 16 output jacks as I believed reading the other ticket -- indeed a quick search shows it has 8 output jacks, but rather in its ability to mix in hardware; thus Haiku can send up to 12 tracks to the card and offload mixing to the card, instead of mixing in the master mixer and sending that unique stereo feed to the card.

To wit, the functions that setup output for HDA (baseline) and Echo, side by side, do exactly the same thing:

• ​http://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/drivers/audio/echo/multi.cpp?id=2311bdf24f00d956ce57db799321f02c2749a366#n565
• ​http://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/drivers/audio/hda/hda_multi_audio.cpp?id=2311bdf24f00d956ce57db799321f02c2749a366#n141

They create one input and one output (both stereo, so 2x2 = 4 channels total), unlike my HDA hack which works by extending the number of channels from 4 to 6 (for a second stereo output).

So the extra "busses" in the Echo driver are routed to the Echo mixer, not to the extra Echo jacks; unlike the HDA hacked driver which does route the (pre-mixed) multi audio to multiple output jacks.

==

Also tried the "Output/Input mapping" hidden tabs in Media Preferences, but it looks like they deal exclusively with output0, not with the output1 I create in the HDA hack. So it probably does 'stuff' when run with an Echo, but not for us with the HDA hacked driver. (to wit: in the Output mapping tab, none of the checkboxes have any effect on output1, and for output0 only 2 of them all have an effect; same with Input Mapping tab).

So looks like we are trail-blazing with HDA in this ticket, it will be the first driver to really address several jacks :-)..

(I still find the "Routing" tabs are a possible inspiration for the GUI to create in the future to expose HDA multi-output capabilities though).

[pulkomandy]

Removing HDA audio tickets from Beta2 milestone, no one has worked on it and opensound is available at least as a stopgap.