[PM] cannot override 'system' driver

[ttcoder]

This is with the 21st of Sep. PM-branch build (a few days before the merge). Can try with a more recent nightly if necessary.

I have a (pre-built) version of the hda driver that I copy to new installs of Haiku to make them "multi-out-jacks" aware (pending integration of my changes some day). I have a 3 line script that shuts down media_server, unzips the driver, relaunches media_server ..etc.

I tried to do the same thing as usual in a PM nightly, just adding non-packaged to the config/non-packaged/add-ons ..etc filepath to unzip the driver to, and... it made the system unbootable, block on the third icon at boot, consistently (found out that if I enable "on screen debug output" it allows to boot to the desktop, so maybe the problem is tied to the weird timing problems I've had on this Asus F2A55M mobo earlier).

There are separate issues I should file about this maybe (should the Migration wiki document explain how to do this correctly, maybe non-packaged is not the right place to put it? or did I do it correctly but there's an issue with the hda driver ?), but in this ticket I'll focus on something that seems worrying at first, in italics for emphasis :-).. :

Booting from another partition, I have removed the hda driver and its audio/hmulti/hda symlink and Haiku is _still_ unbootable

Ok so the package code is not state-less, no problem, find the "state", right ?. So I Iooked into "administrative" directories ..etc and did not see anything that could explain it.. So the system is back to what it was (in appearance at least) before I changed the driver, and yet it still refuses to boot up. What am I missing ? Something's damaged or corrupted by my driver "upgrade", in the packagefs maybe ?

[ttcoder]

Replaced the /boot/home folder with the nightly image's one and I can boot up without safe mode options again.. Though as I dig further (doing "dichotomic rebooting") tonight I realize it might just be that my boot timing issues are back; will keep this ticket posted.

[axeld]

IIRC the kernel does not support one driver with the same name overriding the other, so maybe both are being loaded.

[bonefish]

Replying to axeld:

IIRC the kernel does not support one driver with the same name overriding the other, so maybe both are being loaded.

Oh, I wasn't aware of that. But it would be desirable, no?

[ttcoder]

Go figure:

I've re-added the full previous contents of /boot/home, chunk by chunk, and the system still boots up fine. Boot timing issues are clearly back though. So probably it's more bermuda triangle weirdness from this system.

Now to the interesting bit:

I've even tried to re-add the hda driver build, and the system boots up fine, so the whole premise of the ticket lays on nothing it seems... However, the hda driver is completely ignored.

Only the 'stock' hda driver runs (see syslog to be attached soon). Running pkgman search hda yields no result, so obviously the drivers live in haiku.hpkg or something.

It will be kinda hard for driver developers to develop drivers if they can't uninstall system ones and can't overrule them either by placing newer ones in home/config ;-)

I'll wait a few more days for the nightlies to mature up (and maybe rebuild my 'custom' hda driver from source, though it ought to be binary compatible) before I start pinging again :-)

@axel: used to work, that's how I worked on the hda driver, before the PM merge ..?! So what is the approved way of adding a driver to home/... I thought it was possible to add drivers in home/, even if they 'override' system ones ? Or do you have to delete the system one first ? (hard to do now that system is read-only ;-)

@bonefish if you read this, you might want to add a "Changes for driver developers" in this document (if that is indeed the right place): https://dev.haiku-os.org/wiki/PackageManagement/Migration , for people wondering how to remove a system driver (since it seems they are now monolithic with the whole OS, cannot be "detached" from the rest?) and how to override a system driver with another version of it, in home/ or elsewhere.

Or maybe I'm just missing something obvious...

[ttcoder]

Haiku ignores my driver and runs the 'stock' hda driver; with a terminal session appended at bottom..

[ttcoder]

Ok so on a hunch I typed ls /dev/audio/hmulti/hda and in there I have ... two entries, 0 and 1... <insert astonished smiley here> ..! [EDIT: so axel had quite the right suspicion]

So which one gets [accessed] by the media_server (obviously the wrong one) and how do I change the order of precedence? :-P

[korli]

Replying to ttcoder:

Ok so on a hunch I typed ls /dev/audio/hmulti/hda and in there I have ... two entries, 0 and 1... <insert astonished smiley here> ..! [EDIT: so axel had quite the right suspicion]

You've two entries because there are two HD audio devices in your system (the graphics card exposes a HDMI audio controller too).

I dunno whether overriding a system driver with a user driver has ever worked before. Does ~/config/non-packaged/add-ons/kernel/drivers/bin/ ends up in /system/add-ons/kernel/drivers/bin/ correctly?

[ttcoder]

Oops my lack of experience with this motherboard is showing.

Seems the answer to that question is, it does not, there is only one hda instance and its date is somewhat consistent with the nightly's date but not my driver's build:

~> ll /system/add-ons/kernel/drivers/bin/hda 
-r-xr-xr-x 1 user root 55939 2013-09-22 03:59 /system/add-ons/kernel/drivers/bin/hda
~> ll config/non-packaged/add-ons/kernel/drivers/bin/
total 72
-rwxr-xr-x 1 user root 71595 2012-11-17 01:03 hda
~> uname -a
Haiku shredder 1 c996b65 Sep 21 2013 21:47:23 BePC Haiku

With what I understand of PM thus far, it seems the only way to use/test a custom build of a driver is to rebuild the 50 MB haiku package that contains it, as per this listing:

~> package list /system/packages/haiku.hpkg | grep hda
        hda                          55939  2013-09-22 03:59:55  -rwxr-xr-x
            hda                          0  2013-09-22 03:59:55  lrwxr-xr-x  -> ../../../bin/hda
~> 

I'll try to do a script that does this automatically (package extract to /tmp, cp mybuild/hda /tmp/whatever/hda, package create, mv mycreatedpackage /system/packages/haiku.hpkg).

If that works that will be a valid (if clunky :-) work-around for the problem...

This machine is being lent to a customer in 2 days, so I might do my attempts on another one, but shouldn't change the equation much.

EDIT: I don't suppose it would help the driver be accepted by the system, if it's built with gcc4 instead of gcc2?

[korli]

A suggestion would be to build a separate package which once activated would override the driver. I dunno exactly the policy for colliding files between packages, maybe the latest modified.

[ttcoder]

Giving up for now.. Here's the latest..

Welcome to the Haiku shell.

~/Desktop> package create -C hda_new_package/ hda2.hpkg
----- Package Info ----------------
header size:                     80
heap size:                    30391
TOC size:                      1377
package attributes size:        127
total size:                   30471
-----------------------------------
~/Desktop> package list hda2.hpkg
package-attributes:
        name: hda2
        summary: hda multi
        description: hda patched to provide multiple outputs/jacks
        vendor: hda2
        packager: ttcoder
        flags:
                system_package
        architecture: x86_gcc2
        version: 1.0-1
        copyright: unknown
        license: MIT
        provides: hda
        replaces: hda
add-ons                                  0  2013-10-02 12:50:28  drwxr-xr-x
  kernel                                 0  2013-10-02 12:50:46  drwxr-xr-x
    drivers                              0  2013-10-02 13:07:56  drwxr-xr-x
      bin                                0  2013-10-02 12:50:28  drwxr-xr-x
        hda                          71595  2012-11-17 01:03:01  -rwxr-xr-x
      dev                                0  2013-10-02 13:08:01  drwxr-xr-x
        audio                            0  2013-10-02 13:08:07  drwxr-xr-x
          hmulti                         0  2013-10-02 13:08:15  drwxr-xr-x
            hda                          0  2013-09-30 18:40:43  lrwxrwxrwx  -> ../../../bin/hda
.PackageInfo                           243  2013-10-02 13:34:53  -rw-r--r--
~/Desktop> mv hda2.hpkg /system/packages/
~/Desktop> pkgman search hda
Installed  Name  Description
----------------------------
system     hda2  hda multi  
~/Desktop> ll /system/add-ons/kernel/drivers/bin/hda
-r-xr-xr-x 1 user root 55939 2013-09-22 03:59 /system/add-ons/kernel/drivers/bin/hda
~/Desktop> echo argh, still not overriden
argh, still not overriden
~/Desktop> 

Also tried...

• rebooting
• using "freshens", or even "conflicts", instead of replaces

But nothing changes the mtime of the "hda" system file...

If we're lucky, I just missed (yet another) trick in .PackageInfo and someone will point it out..

If we're unlucky it's actually impossible to update drivers on an existing hrev and driver development now requires rebuilding a whole haiku distribution, but I'd be shocked ;-)

[korli]

Replying to ttcoder:

hda 71595 2012-11-17 01:03:01 -rwxr-xr-x

..

But nothing changes the mtime of the "hda" system file...

I think you should touch your hda file to something newer than 2013-09-22...

If we're lucky, I just missed (yet another) trick in .PackageInfo and someone will point it out..

I think "freshens" should target what haiku.hpkg provides (I guess "haiku"), not hda.

If we're unlucky it's actually impossible to update drivers on an existing hrev and driver development now requires rebuilding a whole haiku distribution, but I'd be shocked ;-)

We shouldn't have to replace a driver in the first place, either override in home or patch the driver upstream. What's this hda patch anyway?

[bonefish]

packagefs itself doesn't care about "freshens", "replaces", etc. That's a hint for the package dependency solver only.

As Jérôme writes, your hda file is older than the one in haiku.hpkg. Currently only the modification time determines which file is chosen.

[ttcoder]

It works! After doing a 'touch' indeed.

Thanks both for being persistent, missed it first time korli mentionned it...

FWIW this is a build of hda with ​this hack included, so that the half-dozen jacks at the rear of the computer get alloted to 2 different outputs instead of just 1. Some day I'll resume the discussion in that ticket, to find a proper way to code it so that it's useful to the general public, my hack is probably too specific to us.

As to this one ticket.. Guess it's time for "curtains", especially as it's not clean (I've mixed up two different issues, bad habit). There was a few seconds of (hair rising :-) wait before package_daemon woke up and changed the mtime, I've had to reboot after that as a media_server restart was not enough for the second output to show up (I gotta try to restart media twice tho, I seem to recall that was needed last year), and I have yet to formalize this driver 'upgrade' script into a clean Jamfile, but none of the above is a good reason to keep this one ticket open.

[ttcoder]

Thinking of it more.. There might be more to this discussion before closing the ticket.

What if I have a system component (e.g. driver, app, anything) override, that I want to distribute with my app. Contrarily to things that are under my control (e.g. upgrading to successive versions of my apps), I cannot predict the mod-time of the component I want to override: if the client has a recent nightly of Haiku, more recent than the nightly I use for development, all the components inside will have a higher mod-time than the overriden components I'm packaging.

To ensure proper override, I could do touch --date=January,1,2099 <file> on all concerned 'overrides' and be good for close to a century, but that's a bit kludgy...

But maybe the PM philosophy is that overriding the system is discouraged, and so it should only be done with the above 'touch' rather than made easy with an explicit mechanism?

Just food for thought.. Me I'm happy as long as there is an easy workaround anyway, and 'touch' is easy indeed.

[ttcoder]

Maybe this can be closed since Ingo has commited hrev46391 (didn't try it yet, but normally if the 'stock' hda driver can be disabled at the packagefs level, that addresses comment:2 and the customized hda driver in non-packaged will be loaded instead, seems like the perfect solution).

[bonefish]

Indeed, that's the kind of situation that the package entry blacklisting is meant for.