USB isochronous streams

[wkornewald]

Our USB stack needs to support isochronous streams.

[korli]

Shouldn't this be reassigned to emitrax ?

[emitrax]

The stack itself now has isochronous support, I'd split it into two different enhancement tickets, one for UHCI (which has not-well-tested support) and one for EHCI. What do you think?

[mmlr]

I'd leave it like this. It serves as a reminder and I think we can keep track of the three points it combines.

[siarzhuk]

I have temporarily taken ownership of this to let all know that I'm working ATM on implementation and testing isochronous support for OHCI controllers.

[modeenf]

Replying to siarzhuk:

I have temporarily taken ownership of this to let all know that I'm working ATM on implementation and testing isochronous support for OHCI controllers.

Nice. Still working on this?

[siarzhuk]

Replying to modeenf:

Nice. Still working on this?

Yes. Hunting for problems and trying to bring the usb_audio to life. ​http://siarzhuk.dyndns.org/cgit/haiku.zz/log/?h=isochronous

[modeenf]

Nice.. I will save that link ;)

If I have understode isochronous. Isochronous are needed for streamin over USB. Then Bluetooth and usb webcam need's it?

[siarzhuk]

Replying to modeenf:

Nice.. I will save that link ;)

If I have understode isochronous. Isochronous are needed for streamin over USB. Then Bluetooth and usb webcam need's it?

It depend on the nature of transferred data. Isochronous is intended for time-critical faults-tolerant data streams. Typical samples of such data are audio or video streams that can lost some part of frames repeatedly without affecting on subjective data consistency. AFAIK Bluetooth provides very wide set of data transfer models. Some of those models can be faults-tolerant (audio-data) some will require data safety.

You can use the listusb -v to get info about your bluetooth device - in case there are some isochronous endpoints listed - you will need this support form the stack.

[siarzhuk]

Good news: OHCI has isochronous support starting from hrev45972. Unfortunately both UHCI and EHCI implementations need to be redesigned to be triggered by HC's interrupts instead of using additional isochronous transfers finishing thread that eats 100% of CPU. Additionaly EHCI HCD implementation uses so agressive locking algorithm so TransferCallback cannot be used to submit the next isochronous transfer because of corresponding double lock panic. I hope to look into UHCI implementation in the nearest future.

[korli]

(In #9933) I removed the relation with UVC and isochronous streams as these devices use a vendor specific bulk-only interface.

[siarzhuk]

(In #10521) Thanks for feedback. You just lucky owner of the OHCI USB controller. Haiku has correct implementation of isochronous transfers for it. UHCI, EHCI and XHCI need to be refactored as described in #1045. Using usb_audio with such controller types crashes the system. That's why it is not in the image yet.

[anevilyak]

(In #11256) As those two tickets already address everything mentioned in this one, there's no reason for it to exist.

[danboid]

The main and only real OS/driver level issue that prevents me switching to Haiku from Linux is that I cannot use my USB audio devices (such as the Focusrire Scarlett 18i20 and 2i4) via EHCI.

I have a few questions for siarzhuk I hope he's kind enough to answer:

• Will the Haiku USB audio driver get enabled when we have isochronous streams working under EHCI or will we have to wait for XHCI to work too before that happens? The latter would seem silly to me. Is there even such a thing as a USB 3 only audio device?

• Does the Haiku USB audio driver support audio input, unlike OSS? Should it be able to recognise and use all 8 XLR inputs on my 18i20? Both my Focusrite devices are USB class-compliant and work great under ALSA/Linux.

• Is there any support for S/PDIF IO in the USB audio driver yet?

Thanks!

[siarzhuk]

Replying to danboid:

• Will the Haiku USB audio driver get enabled when we have isochronous streams working under EHCI or will we have to wait for XHCI to work too before that happens? The latter would seem silly to me.

As already described somewhere in comments above - the reason not including usb_audio in the image - the behaviour is just dangerous on systems with incomplete isochronous transfers support (EHCI, UHCI) - system locks up or goes to KDLs. But on OHCI controllers it has worked for me well. So there are no special requirements for XHCI support for including usb_audio as soon as UHCI and EHCI will be finished.

Is there even such a thing as a USB 3 only audio device?

No idea about current state - but besides of existence of USB Audio 2.0 specifications for many years - most of hardware still produced with USB Audio 1.0 specs. IMO we have another decade before USB Audio 3.0 specs will be written and such hardware will be produced. :-) usb_audio was designed to support both specification 1 and specification 2. But the usb audio 2 devices support was obviously not tested.

• Does the Haiku USB audio driver support audio input, unlike OSS? Should it be able to recognise and use all 8 XLR inputs on my 18i20? Both my Focusrite devices are USB class-compliant and work great under ALSA/Linux.

AFAIR, as soon as those inputs are published by device in standard USB Audio compliant descriptors they will be available for driver - and, hopefully, for user too.

• Is there any support for S/PDIF IO in the USB audio driver yet?

AFAIR it depends on support of such input type in Media Kit. At the moment of active usb_audio development very basic set of input types was available and supported by Media Kit.

[danboid]

Thanks for your quick reply siarzhuk!

It seems I misunderstood the state of the USB audio support so I need to slightly rephrase one of my questions.

Presuming that EHCI and UHCI are separate code entities that aren't going to be solved in one hit somehow, could we see USB audio support enabled in Haiku if isochronous works for EHCI but not UHCI? UHCI and OHCI are rare on machines from the last decade or so which mostly have EHCI and/or XHCI. Could we not just disabe USB audio if UHCI is detected if that was the case?

Sounding promising!

[danboid]

Have you tested audio input under OHCI?

[siarzhuk]

Replying to danboid:

Presuming that EHCI and UHCI are separate code entities that aren't going to be solved in one hit somehow, could we see USB audio support enabled in Haiku if isochronous works for EHCI but not UHCI? UHCI and OHCI are rare on machines from the last decade or so which mostly have EHCI and/or XHCI.

To work with USB 1.0 hardware (that is most of USB audio devices until now, BTW) USB 2.0 controller should have either companion USB 1.0 controllers (OHCI or UHCI) to delegate them functions or more complicate hardware. The companions solution, AFAIR, was very popular until USB 3.0 has seen the light. So most of PCs with USB 2.0 should have either OHCI or UHCI controllers onboard too.

Could we not just disabe USB audio if UHCI is detected if that was the case?

Regardless if this is controlled by OHCI, UHCI or EHCI the driver works with generalized intefraces and have no info about the flavour of USB controller. "Dirty hack" solutions have no chances to pass through our strict Code Quality Control Check. ;-)

Replying to danboid:

Have you tested audio input under OHCI?

#9951 is about it. ;-)

[X512]

Any progress here?

[waddlesplash]

Yes, the EHCI driver now supports inbound isochronous transfers (not outgoing ones, though, which are more useful for USB audio), and the XHCI driver supports bidirectional isochronous transfers (however, this code is not very well tested as the USB audio driver needs a lot of work, and my webcams are not compatible with the USB Webcam driver it appears.)

If someone works on the USB Audio or USB Webcam and finds bugs in the XHCI iso implementation, I will be happy to work on them.

[waddlesplash]

FWIW, in my testing the USB Audio driver was flaky even under QEMU's emulated USB audio controller. It may be worthwhile to start from that instead of on bare metal.

[rudolfc]

Hi waddlesplash, I'm curious if you own usb2.0 external audiocards? I've got two over here which I use to make music via Jamulus. These are both 2.0 spec, and do not work yet with usb_audio.

I am tweaking a bit with that driver to see if I can get sound out of one of those, but it turns out probably a bit serious work has to be done for the 2.0 spec. Testing on EHCI upto now, and indeed I just stumbled upon the double locking panic (though outbound transfer I just saw are not yet supported on EHCI)

Maybe most work needs to be done on the samplerate stuff since usb audio now use clock sources with multipliers and divs to set the rate somehow, I don't understand that yet.

Using a behringer U-phoria UMC202HD (costs around 65 euro's, german brand, quite good actually) I get input sliders after tweaking the driver, and also output buffers are generated and tried to exchange (which results in the panic).

Using a Focusrite Scarlet 2i2 third gen, the inputs are still missing, but outputs behaves the same it seems. This card costs around 200 euro's, neat looking, works good, but has a tad more latency than the very quick behringer..

Would you have any plans to do some more work on usb_audio? Or the USB drivers for isochronous transfers?

[waddlesplash]

I don't have any USB2 audio devices, no. At present, USB audio is rather far down on my list, I'd be more than happy to fill you in on what I know if you'd like to take a look at it (do find me on IRC.)

I don't know much about EHCI, so I can't help there, and I haven't tested anything beyond USB audio with the experimental isochronous support in XHCI. It is more than possible that some or more than just some of the USB audio problems are because XHCI does not implement isochronous support properly. I'm happy to help with the stack end of things, I just didn't go further there because I didn't have anything (or anyone) to test with.

FreeBSD appears to have a "uaudio" driver which supports both 1.0 and 2.0 devices, you might take a look at that.

[rudolfc]

Thanks for the hints and help offer, I appreciate that :-)

I just tried the audio 0x0200 type interface on the xhci driver usb on my newer skylake system, but this is not working at all yet, on a rather basic level maybe, and maybe I am seeing the same problem (in theory) as I saw on my microsoft surface laptop (LP version about same age intel chipset, I expect also xhci):

There seems to be a power problem on both systems using USB. With a rather 'light' load connected things work OK, but if I increase load either by connecing a hub externally in between, or by using a single device that consumes a bit more power (older USB stick, or bigger audio interface so to speak), USB fails.

On skylake, when I connect the audio interface, syslog shows me that a device 'just has been -dis-connected. Then it tries to connect after all, fails, and the cycle repeats indefinately.

UPDATE: it fails during first message processing: either the checksum is wrong, or the number of received bytes is too low/wrong (should be 8 I think I saw in the code) (pointing at the same direction, power, possibly).

Looking at the code it seems USBhub is where it happens: there's a piece of code that waits for the power to stabilize, and I think this is where a general problem is still sitting, on at least both of my mentioned systems.

I'll try to fiddle a bit with that, thought first attempts failed I expect because my sourcecode base I am compiling is too new compared to the running version of Haiku, so I need to upgrade first and retry.

Oh UPDATE2: When I connected the audio interface pre-boot, the repeating thing does not happen. Since I boot from USB using xhci also, I expect things are OK (I can't test further yet since also media services refuse to run on this system still): again, this -could- be power (edge) related.

I'll try to come to a conclusion about this and report back..

[Starcrasher]

Have a look at #17505, I added datasheet for C-Media CM108B which is used in USB headsets.

Maybe there are some useful infos in that could be useful?

[rudolfc]

Hi waddlesplash!

I located two minor faults in the XHCI driver:

• Indeed there's a timing fault, the audio interface need more time to 'spin up' before you communicate with them. please add a simple spin() instruction at line 1549 in xhci.cpp just before the device descriptor is fetched. I tried 100uS which is not enough. 500uS makes both sound cards successfully be configured, so my advice would be 1000uS (tested OK as well) so devices even a bit slower starting up will work (I will most definately try this later on on my microsoft surface 3 which is flaky, maybe this fixes it there too :-) On a sidenote: the TRACE line apparantly also costs this amount of time (> 100uS) as enabling that one line 'fixes' the problem as well.
• there's a fault with variables logged which you will not see unless you enable debugging a bit further down in the same file: line 2489, TRACE("HandleTransferComplete... argument 1 and 3 will not compile on 32bit systems.

Next up I'll retry my usb_audio tweaked driver with XHCI on Skylake to see if I can get any actual sound out of these cards. Apparantly the HDAudio driver had some improvements lately as well, since that runs now on this system (so media servers are up too)

Thanks in advance for applying this patch!

[waddlesplash]

Indeed there's a timing fault, the audio interface need more time to 'spin up' before you communicate with them. please add a simple spin() instruction at line 1549 in xhci.cpp just before the device descriptor is fetched.

This is rather strange, we should not need to wait at this point. I'll check the specification, this may be due to SetAddress() not granting enough time, or some other part of the stack (Hub Explore thread?) not having a requisite delay.

Alternatively, what exactly happens? Do we just get a stall? In that case we should retry fetching the device descriptor.

[rudolfc]

Starcrasher, that chip implements 'audio device class specification 1.0' which is supported by usb_audio as is. I'm fiddling with audio device class specification 2.0 compliant cards. An example of a chip that can work with that is (if all is right): ​https://www.cmedia.com.tw/products/USB20_HIGH_SPEED/CM6642.

[rudolfc]

Hi waddlesplash, if no waiting time is here there will be no transfer at all from the looks of it. At least if after transfer which failed I look at the 'received bytes' the count is (still) zero.

I cannot enable full tracing as that renders my system unbootable because I boot from the same controller, and also have a mouse and keyboard connected there.

So, if you think you don't want to go the wait route in the end because the spec says you don't have to wait (beware of specs ;-), maybe it can be added as a temporary workaround until it's more clear what -is- the root cause then. But if you have a few specific tests I could do that's ok too.

[waddlesplash]

Indeed, it looks like there is a snooze() in the USB < 3.0 initialization code: ​https://xref.landonf.org/source/xref/haiku/src/add-ons/kernel/bus_managers/usb/BusManager.cpp#163

The snooze is for 10,000us (10ms). Clearly we should do the same in XHCI. It's interesting nothing else ran into this problem before...

Can you please post a syslog of this failure occurring, so we can see exactly what it looks like?

[waddlesplash]

XHCI sec. 4.6.5 p114 does have a note indicating that "software is responsible" for adhering to this "recovery interval".

I made those changes in hrev55769. I'd still like to see what errors were originally printed on failure, however. Maybe some other open tickets we have were due to this.

[waddlesplash]

Starcrasher, that chip implements 'audio device class specification 1.0' which is supported by usb_audio as is.

It isn't, though. The reason the driver is disabled at present is because it's presently rather broken even with 1.0 devices. For me it has glitchy audio (random breaks every few hundred ms, and on two of my devices, mono output only.) So the driver needs work for 1.0 devices to be usable anyway.

[rudolfc]

I'll upload the part showing the error with default logging. You state nothing else runs into this problem, but I suspect that's just what you saw and/or what was recognized as belonging there. When you are working on the 'interface' between electronics and firm/software these things are more or less daily routine/trouble.. I lost count for instance in how many occasions the specs were inconclusive or just plain wrong (at my work place).

Anyhow, for this case I suspect it's not just the device that is slow, it might also be connected to the power/ramp-up inside the host or it's local power supply and especially the combination of 'slow' devices with such host controller. As stated before, I also see a lot of trouble on my surface 3 which also uses the xhci driver: though at this point it remains to be seen if it would be solved there with the timing update we are looking at. (when the driver is updated I can test that since it's practically impossible to do that with a non-packaged solution I fear which seems needed on UEFI (I might be mistaken in case of usb bus drivers)

BTW I did also modify/test with usb's hub.cpp where I originally suspected the timing error: but delaying there, or just not exec'ing the debounce routine did not solve the audio card problem.

For usb_audio even the header file we have is still incomplete (USB_audio.h). It does not support v2 audio spec (missing pieces): I had to expand it a little for my tests.

For my test (nasty test, I am learning to understand stuff piece by piece) indeed the usb_audio driver is loaded, buffers are created, (some) controls get created in Media prefs, etc. No audio yet (was to be expected due to new clocking mechanism at least in audio spec v2), and on top of it all the usb bus hangs (until I unplug and replug a few times). I'll not waste your time with further details and I don't dare sharing this 'code' currently).

Thanks for looking at my report!

Updates:

• For errors in specs I usually see these in chipspecs. That's something else than a protocol spec (though inconclusive probably happens there, or just that manufacturors don't follow that spec, like I saw on I2C, just for the sake of not paying fees to Philips for use for instance)
• Oh yes, ring underrun and ring overrun I saw with usb_audio on the v2 cards, no doubt my own fault..

[rudolfc]

syslog with failing xhci usb driver on audiocards audiospec v2

[rudolfc]

So when I plug in the audiospec v2 audiocards, most of the time I first see a disconnected message, then the endless loop of messages you can recognize here are dumped, like so..:

KERN: usb hub 2: port 1: device removed

KERN: usb hub 2: port 1: new device connected

KERN: usb xhci 0: transfer error on slot 4 endpoint 1: USB transaction

KERN: usb error xhci 0: failed to get the device descriptor: Device check-sum error

KERN: usb xhci 0: transfer error on slot 5 endpoint 1: USB transaction

KERN: usb error xhci 0: failed to get the device descriptor: Device check-sum error

KERN: usb hub 2: port 1: new device connected

KERN: usb xhci 0: transfer error on slot 6 endpoint 1: USB transaction

KERN: usb error xhci 0: failed to get the device descriptor: Device check-sum error

...

UPDATE: forget the last message being connected: that was my usb stick being plugged in to fetch the syslog :)

[rudolfc]

Hi, so the nightly was already updated. I just booted the microsoft surface 3, with external HUB, external netwerk card (USB) and external usb keyboard.

Super (and never seen such a rapid Haiku boot at that)!! That was the same timing problem :-) This means I can start fiddling on this system with intel graphics, LP version when I have time..

Update: even the USB ethernet adapter just works..

Update2: please note that using more than only this same usb stick made it impossible to boot before. Even only the usb stick connected was already flaky. On Icon 4 in the bootscreen the result was always that no boot volume was found, which makes sense, as xhci takes over at that point, but had timing issues.

[harshad]

Not sure if this is related. I have been trying to use a Keychron K2 keyboard: ​https://www.keychron.com/products/keychron-k2-hot-swappable-wireless-mechanical-keyboard on haiku. I use it in the wired mode, has a USB C connection. Whenever I start typing, the key gets 'stuck' i.e. keeps repeating and the system becomes unresponsive. Only way to fix it is by removing the cord.

Is there something I could try to get it working? One thing I have observed is under the Input applet, whenever I connect a normal keyboard, it adds only one Usb Keyboard entry. But when I connect the Keychron, 2-3 entries get added.

EDIT: this is on Haiku x86-64 Beta 3.

[anevilyak]

That is unrelated to this ticket, keyboards don't use isochronous mode (it's for streaming devices like webcams).

[bipolar]

Replying to harshad:

I guess that you should open a new ticket for that keyboard issue.

Attach a copy of the output of listusb (with the problematic keyboard connected, possibly *after* you boot up with a "normal" keyboard, so you can actually type that command on a Terminal window :-D).

Also attach a copy of the /boot/system/var/log/syslog (and syslog.old, if any). Maybe they already contain error messages that can help pin-point the problem.

[harshad]

Thanks for the help - ticket #17937 logged for the issue!

Replying to bipolar:

Replying to harshad:

I guess that you should open a new ticket for that keyboard issue.

Attach a copy of the output of listusb (with the problematic keyboard connected, possibly *after* you boot up with a "normal" keyboard, so you can actually type that command on a Terminal window :-D).

Also attach a copy of the /boot/system/var/log/syslog (and syslog.old, if any). Maybe they already contain error messages that can help pin-point the problem.

[waddlesplash]

Following recent refactors, isochronous transfers seem to work OK on XHCI at least. I created #18773 for the UHCI problems, but overall I think we can finally close this.

[danboid]

Well done Mr 'splash!