Opened 21 months ago
Last modified 21 months ago
#18333 new enhancement
device_manager: introduce driver supported device ID registry to speed up boot
| Reported by: | X512 | Owned by: | nobody |
|---|---|---|---|
| Priority: | normal | Milestone: | Unscheduled |
| Component: | System/Kernel | Version: | R1/Development |
| Keywords: | Cc: | ||
| Blocked By: | Blocking: | ||
| Platform: | All |
Description
This is hrev56855.
Currently device manager do a lot of driver probing for finding compatible driver for device node that takes a lot of time and slows down boot. Device manager also contains ugly hard coded PCI device category table (https://git.haiku-os.org/haiku/tree/src/system/kernel/device_manager/device_manager.cpp?id=b6356b9116f2238d880e558637200cde1cd7e407#n1633) that attempts to reduce probe scope, but it is not really suitable for non PCI devices and often need adjustments when adding drivers for non-PCI hardware.
The solution is storing driver compatible hardware IDs statically, for example in driver add-on resources. So it would be possible to quickly lookup compatible driver without loading and calling probe code for a lot of drivers.
Kernel can collect (and cache if needed) compatible hardware IDs information for all available drivers into central kernel registry and provide an interface to lookup most suitable driver add-on for specified device node attributes.
Device compatible hardware information can be represented as a list of device node attributes needed to match, for example:
device/bus = fdt,
or [
{fdt/compatible = riscv,plic0}
{fdt/compatible = sifive,fu540-c000-plic}
{fdt/compatible = sifive,plic-1.0.0}
]
Attachments (4)
Change History (25)
comment:1 by , 21 months ago
| Summary: | device_manager: introduce driver suppirted device ID registry to speed up boot → device_manager: introduce driver supported device ID registry to speed up boot |
|---|
comment:2 by , 21 months ago
comment:3 by , 21 months ago
Possible driver compatible hardware IDs definition language:
driver module busses/ata/legacy_sata/driver_v1 attrs device/bus pci device/vendor device/id driver attrs ; VIA device/vendor 0x1106 device/id 0x3149 0x3249 0x0591 driver attrs ; ALI device/vendor 0x10b9 device/id 0x5289 0x5287 0x5281 driver attrs ; NVIDIA device/vendor 0x10de device/id 0x008e 0x00e3 0x00ee 0x0036 0x003e 0x0054 0x0055 0x0266 0x0267 0x037e 0x037f 0x03e7 0x03f6 0x03f7
driver driver module busses/i2c/pch_i2c/pci/driver_v1 attrs device/bus pci device/vendor 0x8086 device/id 0x02c5 0x02c6 0x02e8 0x02e9 0x02ea 0x02eb 0x06e8 0x06e9 0x06ea 0x06eb 0x0aac 0x0aae 0x0ab0 0x0ab2 0x0ab4 0x0ab6 0x0ab8 0x0aba 0x1aac 0x1aae 0x1ab0 0x1ab2 0x1ab4 0x1ab6 0x1ab8 0x1aba 0x31ac 0x31ae 0x31b0 0x31b2 0x31b4 0x31b6 0x31b8 0x31ba 0x34c5 0x34c6 0x34e8 0x34e9 0x34ea 0x34eb 0x43ad 0x43ae 0x43d8 0x43e8 0x43e9 0x43ea 0x43eb 0x4b44 0x4b45 0x4b4b 0x4b4c 0x4b78 0x4b79 0x4b7a 0x4b7b 0x4dc5 0x4dc6 0x4de8 0x4de9 0x4dea 0x4deb 0x51c5 0x51c6 0x51d8 0x51d9 0x51e8 0x51e9 0x51ea 0x51eb 0x54c5 0x54c6 0x54e8 0x54e9 0x54ea 0x54eb 0x5aac 0x5aae 0x5ab0 0x5ab2 0x5ab4 0x5ab6 0x5ab8 0x5aba 0x7a4c 0x7a4d 0x7a4e 0x7a4f 0x7a7c 0x7a7d 0x7acc 0x7acd 0x7ace 0x7acf 0x7afc 0x7afd 0x7e50 0x7e51 0x7e78 0x7e79 0x7e7a 0x7e7b 0x98c5 0x98c6 0x98e8 0x98e9 0x98ea 0x98eb 0x9d60 0x9d61 0x9d62 0x9d63 0x9d64 0x9d65 0x9dc5 0x9dc6 0x9de8 0x9de9 0x9dea 0x9deb 0xa0c5 0xa0c6 0xa0d8 0xa0d9 0xa0e8 0xa0e9 0xa0ea 0xa0eb 0xa160 0xa161 0xa162 0xa2e0 0xa2e1 0xa2e2 0xa2e3 0xa368 0xa369 0xa36a 0xa36b 0xa3e0 0xa3e1 0xa3e2 0xa3e3 driver module busses/i2c/pch_i2c/acpi/driver_v1 attrs device/bus acpi acpi/hid INT33C2 INT33C3 INT3432 INT3433 INT3442 INT3443 INT3444 INT3445 INT3446 INT3447 80860AAC 80865AAC 80860F41 808622C1
by , 21 months ago
| Attachment: | usb_serial.rdef added |
|---|
by , 21 months ago
| Attachment: | legacy_sata.rdef added |
|---|
by , 21 months ago
| Attachment: | pch_i2c.rdef added |
|---|
comment:4 by , 21 months ago
- What are the speed gains expected?
- How is backward-compatibility handled?
- What are the risks associated with this change?
comment:5 by , 21 months ago
X512 also made a post to the mailing list about this here, where some of those questions are answered: https://www.freelists.org/post/haiku-development/Driver-supported-hardware-definition-format-proposal
follow-up: 12 comment:6 by , 21 months ago
I think the proposed format is not a good idea, for a variety of reasons.
The biggest one is size, performance, and discrepancy. The format has strings for each and every ID. Thus instead of a mere 4-8 bytes for a PCI vendor/ID, we instead have 13 bytes for "device/vendor", another 9 for "device/id", and then however many bytes of overhead to specify "int32 type" in a packed BMessage (1 byte?) followed by the value (4 bytes each), for a total of 31 bytes. Some of this is saved in the case where there are many IDs with one vendor, but you get the idea. When you have drivers with dozens, hundreds of PCI IDs they handle (like ipro1000), that will mean hundreds of string comparisons and thousands of bytes.
String comparisons and data storage are not cheap. This may wind up adding double-digit percentages to the size of some drivers, even in a packed BMessage format. Further, if one of the purposes of this is to reduce load time, well, one of the reasons driver loading takes time is because of ELF symbol resolution ... which is a lot of string comparisons. Adding hundreds of string comparisons to save others (thousands?) sounds like a bad idea, and increasing space besides sounds even more questionable.
Most drivers likely will want to re-probe the device anyway as it may be somehow categorized into a "family" within driver structures, they may not be able to just take the device manager's probe result directly and start attaching.
By "discrepancy", what I mean is that in this proposal there would be 2 different device ID tables: the ones in the driver source code (because there's just no way to get rid of those, at least for FreeBSD drivers and the like), and the ones in the attributes/packed message. Even if we generate these at build time, it's still duplicated data. If we don't generate these at build time, well, that will be a source of bugs if they can get out of sync.
comment:7 by , 21 months ago
I think it is worth looking at what FreeBSD does here. They have a macro called MODULE_PNP_INFO, which takes a set of parameters pointing to the driver-internal structure of IDs or other attachment information.
Here's the realtekwifi driver, for PCI: https://github.com/haiku/haiku/blob/4f97a431b966877dc688abbe106a2967a2c93653/src/add-ons/kernel/drivers/network/wlan/realtekwifi/dev/rtwn/pci/rtwn_pci_attach.c#L790
For USB it is even simpler, there is a "usb_host_id" structure that is passed to a macro that takes only one parameter (and automatically fills out the MODULE_PNP_INFO parameters.)
The driver-internal table of IDs is thus not duplicated, but its final position within the driver binary is then noted. It is clear how we could use this on Haiku: at build time, specify the MODULE_PNP_INFO-noted region ranges of the driver binary into an extended attribute (i.e. not their contents, just offset+size into the binary.) Then the kernel can read the attribute, then read the driver binary to get the table (which is a C structure, thus packed, and no string comparisons needed) and then performs comparisons as appropriate.
That method would have the advantage that no information is duplicated between the driver binary and some alternate message, and also that parts of the driver ELF will already have been read into disk cache (as opposed to some out-of-band message, which will likely only be read once per boot) when we go to load the full driver binary.
comment:8 by , 21 months ago
So, to summarize: I really don't like the proposed, string-heavy format (and I think that the current device manager is so string-heavy is one of its major downsides; I would favor a redesign that moved a lot of the device ID management into the bus managers, and let them compare or otherwise decide on probe matching; that way PCI, USB, and FDT which all have very different ID schemes and typical matching schemes can all be handled differently.)
But I do think this idea has some merit to it nonetheless, and that FreeBSD and others have something similar we can learn from.
comment:9 by , 21 months ago
I think the current proposal is also over-simplifying things by expecting that most things will be matched by vendor and device ID. There are many places where this is not the case, for example:
- usb_serial for ACM devices (there is a partial example of rdef file for usb_serial but it omits that part)
- NVMe
- usb_audio
- usb_rndis
- SDHCI
My impression from the drivers I worked on (it may just be that I picked the wrong ones) is that matching specific devices/vendor ID is not the main way to match devices, and is hopefully going away as things get more standardized. While the device_manager is not so great for other reasons, at least it makes this easy.
I have to be convinced that a complex thing such as matching a driver to devices can be done with only a static data structure and no code from the driver loaded and run. I don't see how it can be done, but maybe you are smarter than me in designing such a thing. In any case, I would consider the problem differently, by still loading and probing the drivers as we do now, whenever there is a device added/removed or an updated driver. And cache the results of the probing so we don't have to do it for the next boot everytime. This allows to do the work once and store exactly the infos we need so the next boot can be extremely fast, while still allowing any level of complexity in the driver's probing. And it also opens for letting the user manually editing things (to disable a driver, or use two different drivers for two pieces of hardware that would normally be handled by the same driver).
by , 21 months ago
comment:10 by , 21 months ago
I think the current proposal is also over-simplifying things by expecting that most things will be matched by vendor and device ID. There are many places where this is not the case, for example:
Matching can be done by combination of any device_node attributes, not only vendor and device ID. It can be PCI type, sub type, USB class, FDT compatible, ACPI PNP ID etc.
My impression from the drivers I worked on (it may just be that I picked the wrong ones) is that matching specific devices/vendor ID is not the main way to match devices, and is hopefully going away as things get more standardized.
This is how most major OS match drivers. For example Windows use something similar with compatible hardware IDs definitions in driver *.inf file.
NVMe
I see not problems with NVMe matching. Added sample definition that 100% match behavior of existing nvme_disk_supports_device code.
comment:11 by , 21 months ago
I feel that there is some misunderstanding that only "device/vendor", "device/id" attributes can be matched. No, any attributes present in device_node (can be viewed in Devices GUI application) can be used for matching.
comment:12 by , 21 months ago
Replying to waddlesplash:
That method would have the advantage that no information is duplicated between the driver binary and some alternate message
For native drivers matching code can be removed after adding hardware definitions resources. Some additional information such as per device quirks can be stored in info resource property.
Foreign ported drivers such as FreeBSD should not take any effect on Haiku decision making. Haiku is not FreeBSD, please do not make FreeBSD from Haiku. It should be possible to automatically generate resource definitions from ported FreeBSD drivers.
Most drivers likely will want to re-probe the device anyway as it may be somehow categorized into a "family" within driver structures, they may not be able to just take the device manager's probe result directly and start attaching.
Why? Probe code can be fully removed if resource matching is enough. Only code that can't be represented with resource attribute matching should be kept, for example probing that need to actually access hardware.
When you have drivers with dozens, hundreds of PCI IDs they handle (like ipro1000), that will mean hundreds of string comparisons and thousands of bytes.
Messages support arrays of values, so key string will be stored only once for a list of PCI IDs. See legacy_sata.rdef for example. You can compile resource with rc and confirm that key name is stored only once for array of values.
I would favor a redesign that moved a lot of the device ID management into the bus managers, and let them compare or otherwise decide on probe matching; that way PCI, USB, and FDT which all have very different ID schemes and typical matching schemes can all be handled differently.)
It is a bad idea because it will duplicate device matching logic for each bus while it can be done in generic way for all busses. If you think that for some bus attribute matching is not enough, can you give an example?
comment:13 by , 21 months ago
I feel that there is some misunderstanding that only "device/vendor", "device/id" attributes can be matched. No, any attributes present in device_node (can be viewed in Devices GUI application) can be used for matching.
I was just complaining that there was no example of this, now there is. So it helps seeing how it works, and if Waddlesplash wants to provide an alternative way of doing this, maybe with a more efficient data structure, he can keep this in mind too.
The linked FreeBSD driver seems to have a similar limitation: matching is done only on vandor and device id. So, how does FreeBSD handle drivers that work on a device class?
In the world of USB I can probably find examples as crazy and complicated as you want. For example there are some devices that all share the same vendor and device ID: https://github.com/obdev/v-usb/blob/master/usbdrv/USB-IDs-for-free.txt
Or things like the ST-Link v1, which pretends to be an USB mass storage device, but it isn't, and so the Linux USB mass storage driver has a special case to ignore it (and a few other devices): https://sourceforge.net/p/openocd/mailman/openocd-devel/thread/4F5A5693.1040600@freenet.de/ I think our USB HID driver already has some special cases like this as well.
So, the idea is, yes, you can use some attribute or resource or whatever format to optimize the driver loading. But in the end the driver should always have an opportunity to run its own code to do a final check on wether it can handle a device or not, because there will always be some special case we can't anticipate.
Given this, I am more inclined to preserve the probing as it is (in code) and cache the results, instead of trying to do a data-driven probing that would only do a pre-selection of drivers but still require to run code to do the final decision.
comment:14 by , 21 months ago
In the world of USB I can probably find examples as crazy and complicated as you want. For example there are some devices that all share the same vendor and device ID:
How it will be detected in this case? Actually try to interact with hardware and see that it behaves like expected device?
But in the end the driver should always have an opportunity to run its own code to do a final check on wether it can handle a device or not, because there will always be some special case we can't anticipate.
Yes it can of course device manager supports_device hook still can be used if needed. It will be called only for drivers that passed attribute matching phase. So driver can put hardware IDs matching in resources and do the rest probing in code. No duplication is needed. Or supports_device can be empty for drivers where attribute matching is enough.
Given this, I am more inclined to preserve the probing as it is (in code) and cache the results
I feel caching very unreliable and it will probably bring a lot of tricky bugs. And/or it will not actually work in many cases and will do full probe fallback.
follow-up: 16 comment:15 by , 21 months ago
I think it's a good direction, and there is a) a score value; and b) I'd hope it could probe and have a list of compatible drivers ordered by score (else what is it there for?), and it'd try in score order to attach the driver to the device.
comment:16 by , 21 months ago
Replying to jessicah:
I think it's a good direction, and there is a) a score value; and b) I'd hope it could probe and have a list of compatible drivers ordered by score (else what is it there for?), and it'd try in score order to attach the driver to the device.
Yes, if multiple compatible drivers are found, it will be probed by "score" field descending order. I mentioned that in mailing list message:
float "score": If multiple drivers will be matched for the same device, drivers will be probed in descending order of this field.
comment:17 by , 21 months ago
How it will be detected in this case? Actually try to interact with hardware and see that it behaves like expected device?
As described in the linked page, the devices can be differenciated by looking at string descriptors, for example (there are other cases for some of the IDs, using serial numbers for example):
- The USB device MUST provide a textual representation of the manufacturer and product identification. The manufacturer identification MUST be available at least in USB language 0x0409 (English/US).
- The textual manufacturer identification MUST contain either an Internet domain name (e.g. "mycompany.com") registered and owned by you, or an e-mail address under your control (e.g. "myname@…"). You can embed the domain name or e-mail address in any string you like, e.g. "Objective Development http://www.obdev.at/vusb/".
- You are responsible for retaining ownership of the domain or e-mail address for as long as any of your products are in use.
- You may choose any string for the textual product identification, as long as this string is unique within the scope of your textual manufacturer identification.
- Application side device look-up MUST be based on the textual manufacturer and product identification in addition to VID/PID matching. The driver matching MUST be a comparison of the entire strings, NOT a sub-string match.
Anyway, if in the end the driver has an opportunity to run some code and do final checks, that will be enough. It means the matching from the static data can be a bit more "relaxed" and match more devices than the driver actually supports, but the driver can run some code to do the exact checks needed.
follow-up: 20 comment:18 by , 21 months ago
If ELF loading and symbol resolution is the most expensive part, perhaps we should introduce a simpler/more efficient format for drivers?
comment:19 by , 21 months ago
Without replacing the format, there is quite a lot to be done to improve ELF loading time. How do we do symbols lookup? Do we use the DT_GNU_HASH hashtables to speed it up, for example?
https://flapenguin.me/elf-dt-gnu-hash https://blogs.oracle.com/solaris/post/the-cost-of-elf-symbol-hashing
That second article mentions additional tricks (prelinking and lazy binding) that may also be useful.
comment:20 by , 21 months ago
Replying to tqh:
If ELF loading and symbol resolution is the most expensive part, perhaps we should introduce a simpler/more efficient format for drivers?
Fully loading all drivers just to probe for compatible hardware is bad idea anyway because if Haiku even become mature enough, it may have hundreds of megabytes of driver add-on binaries. For example ported Linux GPU drivers, Nvidia kernel GPU driver, various vendor drivers etc.. Driver architecture should be scalable.
comment:21 by , 21 months ago
I kind of hoped it would spark some thoughts similar to hvif, where the icon is made small enough to fit in the inode, no loading needed.
For drivers if you can look at metadata that reject >90% drivers you would never need to do elf loading or symbol resolution. If you then strip + simplify elf you can gain a lot. Do we need to do full relocation or just enough to do detection.
There are so many interesting things to explore both with or without ELF. I'd probably check what could be done with just a indexed attribute.
There was in the original 2004 PnP manager the possibility to define PNP ID strings for each driver, if I remember well. https://www.haiku-os.org/docs/develop/kernel/obsolete_pnp_manager.html