Package kit boot activation doesn't create users

[kallisti5]

If a package is activated at boot, any required users are not created. If package is activated post-boot... users are created.

~> id alex
id: ‘alex’: no such user
~> id sshd
id: ‘sshd’: no such user
~> mv /boot/system/packages/usertest-1.0-1-x86_64.hpkg ~/
~> id alex
id: ‘alex’: no such user
~> mv ~/usertest-1.0-1-x86_64.hpkg /boot/system/packages/
~> id alex
id: ‘alex’: no such user
~> id alex
id: ‘alex’: no such user
~> id alex
id: ‘alex’: no such user
~> id alex
id: ‘alex’: no such user
~> id alex
uid=1000(alex) gid=100(users) groups=100(users)
~> id sshd
id: ‘sshd’: no such user

This explains why the sshd user isn't created at first (or any) boot, and must be created manually.

[kallisti5]

~> package list /boot/system/packages/usertest-1.0-1-x86_64.hpkg
package-attributes:
        name: usertest
        summary: Testing the creation of alex
        description: Alex is love, alex is life
        vendor: Haiku Project
        packager: Alexander von Gluck IV <kallisti5@unixzen.com>
        architecture: x86_64
        version: 1.0-1
        copyright: 1985-2018 Alexander von Gluck IV
        license: MIT
        URL: http://www.unixzen.com/
        source URL: https://cdn.haiku-os.org/haiku-nightly/arm/haiku_unsupported-nightly-hrev52249-arm-mmc.zip
        source URL: https://ports-mirror.haiku-os.org/usertest/haiku_unsupported-nightly-hrev52249-arm-mmc.zip
        provides: usertest = 1.0 (compatible >= 1)
        requires: haiku>=r1~alpha4_pm_hrev52254-1
        user: alex
                real name: alex user
                home:      /packages/usertest-1.0-1/.self/data/openssh/empty
                shell:     /bin/true
        group: alexgroup
.PackageInfo                           779  2018-08-21 19:34:37  -rw-r--r--

~> cat /Data/haikuports/net-misc/usertest/usertest-1.0.recipe 
SUMMARY="Testing the creation of alex"
DESCRIPTION="Alex is love, alex is life"
HOMEPAGE="http://www.unixzen.com/"
COPYRIGHT="1985-2018 Alexander von Gluck IV"
LICENSE="MIT"
REVISION="1"
ARCHITECTURES="x86_gcc2 x86 x86_64"
SOURCE_URI="https://cdn.haiku-os.org/haiku-nightly/arm/haiku_unsupported-nightly-hrev52249-arm-mmc.zip"
SOURCE_DIR=""

PROVIDES="
        usertest = $portVersion compat >= 1
        "
REQUIRES="
        haiku
        "

BUILD_REQUIRES="
        haiku_devel
        "
BUILD_PREREQUIRES="
        cmd:pkg_config
        "

sshdUserHomeDir="/packages/$portVersionedName-$REVISION/.self/$relativeDataDir/openssh/empty"
PACKAGE_USERS="
        alex real-name \"alex user\" home \"$sshdUserHomeDir\" shell \"/bin/true\"
        "
PACKAGE_GROUPS="alexgroup"


BUILD()
{
        true
}

INSTALL()
{
        true
}

TEST()
{
        true
}

Haiku shredder 1 hrev52254 Aug 20 2018 15:59:03 x86_64 x86_64 Haiku

[pulkomandy]

This prevents ssh from working out of the box in Haiku installs, would be nice to fix...

Tentative scheduling for beta2 hoping someone takes care of it :)

[ambroff]

I started looking into this. I'm still learning how the package system works, but I think I see a few problems right off the bat.

Problem 1: The package daemon tries to load the activated-packages file from the incorrect location when initializing a new volume in ​Volume::_InitLatestStateFromActivatedPackages(). It tries to read /boot/system/packages/activated-packages instead of /boot/system/packages/administrative/activated-packages.

This should probably be something more like this (although I'm not totally sure I'm using the StorageKit API properly):

status_t
Volume::_InitLatestStateFromActivatedPackages()
{
	BDirectory adminDirectory;
	status_t error = _OpenPackagesSubDirectory(
		RelativePath(kAdminDirectoryName),
		false,
		adminDirectory);
	if (error != B_OK)
		return error;

	BPath activationFilePath(&adminDirectory, kActivationFileName);
	activationFilePath.Append(kActivationFileName);

	BFile file;
	error = file.SetTo(activationFilePath.Path(), B_READ_ONLY);
	...

Problem 2: Packages are marked as activated ​if the activated-packages file is not found. In the case of first boot after running the installer, there is no activated-packages file, so all packages are marked as activated even if the activation logic has not been performed.

Problem 3: Nothing performs activation on boot. Even if activated-packages does exist, and a new package has been added to the packages dir to be activated at boot, the activation will not happen until the user ​initiates a new transaction, since that is the only place where activation logic is performed.

The only place where activation will happen automatically is ​in response to a filesystem notification that a package has been copied into the packages directory. Booting up after a fresh install and moving the openssh package out of /boot/system/packages and then moving it back at least gets it far enough to create users.

I'm still trying to understand the code so I may be missing something. Assuming my conclusion above is correct, I think the right thing to do is to 1.) assume no packages are activated if activated-packages is missing and 2.) on boot, for each package not yet active trigger the same code that is executed in ​src/servers/package/Root.cpp in response to filesystem events.

[waddlesplash]

I think the right thing to do is...

From my knowledge of the Package Kit at least, I think your analysis is entirely correct, and actually does not sound that hard to implement, indeed.

[pulkomandy]

Fixes for problems 1 and 3: ​https://review.haiku-os.org/c/haiku/+/2342

However, this does not help, because we are in "problem 2" case for a fresh built image.

I tried to create and commit a transaction to activate the packages, but it doesn't work. The launch daemon is not found then and the system does not boot. You can try this by having an empty (but existing) activated-packages file. I'm sot sure why, I'd think once the transaction is commited, things would work as usual?

[nielx]

Assign to agmsmith/nielx

[AGMS]

Having a look at it...

[nielx]

Any update?

[AGMS]

Still investigating, but not full time. Got the dev system set up, got Haiku compiling, poking through the code, examining the activation state setup (sidetracked with a duplicate packages installed problem), next is getting output during boot (figuring out virtual serial port or screen console). Trying the 2342 version. Then wondering when to do boot activation (boot with everything enabled then after boot fix it up?).

[AGMS]

Also noticed that the intel_microcode package's CPU updater can't find its data file. Perhaps the package with the data isn't mounted that early? Or maybe it needs an absolute path.

[nielx]

An interim-fix could be to create all the required users in the post install script, couldn't it?

We still have some weeks before beta 2, but it seems like solving this at the package kit level may be complicated and something we want to have tested.

[AGMS]

Finally finished other chores, got time to look into this bug this week. I now have a dev VM set up nicely so I can build Haiku and then test with serial output for when it fails to boot. But yes, a post-install script could help, except that post-install is broken for running after a fresh install (there's a function to run the jobs, but it's never called).

[nielx]

I actually mean in the sense of using the ​PostInstallScript to create users in the default packages as a workaround.

In any case, the preferred option is of course to make it work as desired.

[AGMS]

Package which tests creating users and groups and running a post-install script to see if they actually got created. Useful for seeing if first boot does create users.

[AGMS]

Problem 1: Incorrect location of activated-packages file.

It somehow finds the file and uses it. The confusion may be about the error message which is about where it finds the packges, not the control file. No changes needed for this one.

Problem 2: Packages are marked as activated ​if the activated-packages file is not found.

Actually makes sense. Otherwise the only packages available while booting up are the ones which loaded the kernel (SCSI driver and file system). Which is why it would crash later on in launch_daemon if you had an empty activated-packages file. Presumably a sensible set of files was put there during the install and we don't need to worry about dependencies; just activate them all. Need this behaviour.

The other trouble is that sometimes the activated-packages file is deleted by the user as a way to save space, so we can't detect which packages need to do their post install actions. Again activating all of them and not doing installation activities makes sense.

Problem 3: Nothing performs activation on boot.

That is the real problem, for first boot. Can't do it too early, otherwise the OS won't be fully running. So it can't be in the second stage boot loader or the kernel initial package activation.

Change ​https://review.haiku-os.org/c/haiku/+/2342 does find files in the packages directory not known to be activated (you have to have enough files in the initial activation list to be able to boot, then it works), and activates them, adding them to the control file and mounting their package file. But it doesn't run the install stuff.

The current hack is for launch_daemon to detect first boot and run all the scripts in the boot/post-install directory. A better hack would be to do the full set of package install operations for all packages, which includes creating users and running the install scripts more properly (the ones specified in .PackageInfo, not just all ones in the directory).

We could fake it by uninstalling everything and reinstalling. But besides being dicy for running programs, it may need network access.

So, I'm thinking of adding a package kit function to do the post install stuff, without the actual install - creating users and running scripts and that config file creation action and so on for all packages. Should it also do a consistency check? Though that would require network access, which we don't have. Perhaps expose the function as "pkgman firstboot" and have the launch_daemon call that when it detects first boot. Unless I hear otherwise, I'll start working on that.

[waddlesplash]

It somehow finds the file and uses it.

PulkoMandy submitted a patch for this one and it was merged already, yes.

Again activating all of them and not doing installation activities makes sense.

OK, you reasoning here also makes sense.

So, I'm thinking of adding a package kit function to do the post install stuff, without the actual install...

How about this: an existing but empty activated-packages file causes all packages to be activated. This way, if anyone deletes the file, we don't re-activate all packages; but if the file is existant but empty, we do (and of course run post-install scripts, etc.)

Currently, an empty activated-packages file will just break the boot (no packages active, etc.) So adding the special-case logic inside the loader to just load everything in this case, and in package_daemon to activated everything on startup in this case, sounds pretty doable and less "hacky".

[pulkomandy]

Replying to AGMS:

Problem 1: Incorrect location of activated-packages file.

It somehow finds the file and uses it. The confusion may be about the error message which is about where it finds the packges, not the control file. No changes needed for this one.

Yes, this has been solved already.

Problem 2: Packages are marked as activated ​if the activated-packages file is not found.

Actually makes sense.

Yes. The idea of my patch was to differenciate between a completely missing file, and an empty file. Only in the latter case we would have done some activation. But that doesn't work, as you explain below, we need the packages "active" to boot the system at all...

Problem 3: Nothing performs activation on boot.

We could fake it by uninstalling everything and reinstalling. But besides being dicy for running programs, it may need network access.

If fact you don't need to uninstall anything. Because of the empty activated-package files, the package system already knows that the packages are not activated. So, you can just trigger the scripts and user creation for all these packages which are "semi-activated". I think this may even happen automatically when you install or remove any package already?

In any case, I don't see a need to expose this in pkgman. Just have an API in the package kit and have the launch daemon call it on first boot.

[waddlesplash]

I still think the empty file is the best solution here, we just need to tweak the boot loader and packagefs to treat an empty file as if it did not exist, and then package daemon can treat them differently.

[pulkomandy]

We have other ways to detect first boot, it sounds much simpler to use one of those.

[AGMS]

I too think having too many first boot detection mechanisms is messy.

Also, there's the problem of running the install stuff for packages which need a reboot before they are functional. There's some half implemented stuff (RunQueuedScripts) for making a directory of symbolic links to things needing to be run after a reboot if, but it is never actually used, and only deals with scripts. Seems to be related to fVolumeStateIsActive, which gets set to false if a system package is involved (name starts with "haiku"), and a temporary activation file is used with the final one written and presumably only taking effect after rebooting.

I'm thinking of a second control file, call it needs-post-install, which lists the package files that need the extra install steps done after a reboot. The package daemon (or whatever is appropriate) will do the install work for those items after boot. If the file is empty, then all the work has been done. It would be nice if the build process could create that file with a list of the packages being copied to the build image. But if the file is missing, as a fallback, all active packages can have their install steps run. Though if the user deletes the file, the install steps could get run twice.

Anyway, I'll be poking around some more in the boot sequence and package loading code just to see what's there. With a side trip out of curiosity to see how the memory management is set up and how the initial loaded packages persist through kernel startup.

[waddlesplash]

They don't persist through kernel startup. The loader loads the kernel and the "boot drivers", of which the disk drivers and packagefs are a part, and these mount and load the packages for real.

[AGMS]

I'd extend the idea of a file listing packages that need delayed post-install processing to include a magic word like "all" to run post-install for all packages. Then on newly installed systems, that file would be created with the "all" keyword. In normal use, it would be empty, or list the files (usually just system packages) needing post-install after a reboot. Not existing would be equivalent to being empty.

So, which to do? A simpler fix to add an API for doing post-install on all packages, called by launch_daemon on first boot? Or a new "packages-needing-post-install-processing" file in the administrative directory and functioning post-install operations after reboot?

Also, should the user and group creation also be delayed to post-install, or do some packages need the user before they start running? Currently it includes them in the file moving transactions, and even has a rollback where it deletes the newly created users (could that delete a pre-existing user?).

By the way, speaking of roll-back, would it make sense to run the pre-uninstall script to roll back the post-install script?

[nielx]

Beta 2 status meeting: good questions that need more thoughts. The decision is to create a quick fix in the post install script in thebeta2 branch

[AGMS]

Discussion from the Beta2 meeting:

[2020-05-02 16:15] <nielx> Let's move on to https://dev.haiku-os.org/ticket/14382? It's a high priority ticked about the user creation
[2020-05-02 16:16] <nielx> AGMS left a comment on it earlier
[2020-05-02 16:16] <AGMS> Package post-install adding users, etc.  Been busy reading code, mostly done.  Got 3 choices for a solution: 1) quick fix hack up the post install script to add users, 2) Add an API to do post-install for all packages, call it in launch_daemon for first boot, 3) Do it properly, make a list (file) of packages that need post-install operations after reboot, magic keyword "all" for first boot.
[2020-05-02 16:16] *** dorje (~vision@cpe-65-27-104-0.new.res.rr.com) has joined the channel.
[2020-05-02 16:17] <PulkoMandy> I would say 2 is fine, 3 doesn't feel much cleaner to me
[2020-05-02 16:17] <AGMS> Though 3 does fix a bug with packages that need a reboot.
[2020-05-02 16:17] <nielx> I was just about to ask that question
[2020-05-02 16:17] <AGMS> Though that's not too many pakcages.
[2020-05-02 16:17] <nielx> I would imagine any Haiku package would require a reboot
[2020-05-02 16:18] <nielx> (though they don't use post-installscripts right now, do they)
[2020-05-02 16:18] <AGMS> The code looks for Haiku*.hpkg or something like that.  And there may be a flag you can set in the package too.
[2020-05-02 16:18] <AGMS> But other than the OpenSSL package which adds users, this doesn't come up often.
[2020-05-02 16:19] <PulkoMandy> since the quick fix can be done quickly (that's the point right) it can be pushed to the beta2 branch after branching, if the other solution is not ready by then
[2020-05-02 16:19] <waddlesplash> in all honesty, it may be best to just do the hack
[2020-05-02 16:19] <PulkoMandy> and we can hopefully have the real fix in beta3 then
[2020-05-02 16:19] <waddlesplash> 3 shouldn't fix any bugs with reboots
[2020-05-02 16:19] <waddlesplash> users can be created without the package being active
[2020-05-02 16:19] <nielx> waddlesplash: just short term or do you think this will remainan edge case?
[2020-05-02 16:19] <waddlesplash> postinstall scripts can't
[2020-05-02 16:19] <waddlesplash> so this is only a problem with packages initially installed needing it
[2020-05-02 16:20] <AGMS> Quick fix and document it?
[2020-05-02 16:20] *** HaikuUser (~vision@031011179136.warszawa.vectranet.pl) has joined the channel.
[2020-05-02 16:21] <PulkoMandy> there's also the option of not shipping sshd with the release, since it's the only affected package
[2020-05-02 16:21] <PulkoMandy> if people install it later on, it will install properly
[2020-05-02 16:21] <nielx> Is it separate from the ssh client?
[2020-05-02 16:21] <waddlesplash> right
[2020-05-02 16:21] <PulkoMandy> is there a reason to ship sshd by default?
[2020-05-02 16:21] <waddlesplash> nielx: ah, I don't know
[2020-05-02 16:21] <waddlesplash> actually it may not be
[2020-05-02 16:21] <waddlesplash> so that's not great, indeed
[2020-05-02 16:22] <waddlesplash> adding the post-install script as a "hack" seems best, for the release, then
[2020-05-02 16:22] <PulkoMandy> seems not, but we can split the package?
[2020-05-02 16:22] <waddlesplash> how does Linux/BSD/etc. do this?
[2020-05-02 16:22] <PulkoMandy> debian has an openssh_server package
[2020-05-02 16:22] <nielx> waddlesplash: packaging or creating users?
[2020-05-02 16:22] <PulkoMandy> of course you may want the client without necessarily your machine being a server
[2020-05-02 16:22] <nielx> I am guessing for creating users it is during the installation phase, executing post-installation scripts
[2020-05-02 16:22] <waddlesplash> ok, so then creating our own _server subpackage sounds fine
[2020-05-02 16:23] <PulkoMandy> but the post-install script is maybe easier as a temporary fix
[2020-05-02 16:23] <AGMS> Can the Network Prefences handle sshd not being present?
[2020-05-02 16:23] <waddlesplash> no idea
[2020-05-02 16:23] <waddlesplash> but it should
[2020-05-02 16:24] *** HaikuUser has left the channel.
[2020-05-02 16:24] <PulkoMandy> ideally the network preferences add-on would be shipped with the openssh_server package. Or at least I think that was the idea for the add-ons based preferences panel
[2020-05-02 16:24] <AGMS> Or make the preferences a dependency?
[2020-05-02 16:25] <PulkoMandy> yes but that's micro-packaging things
[2020-05-02 16:25] <AGMS> Hmmm, uninstalled openssh and network prefs says it is ON.
[2020-05-02 16:25] <nielx> It feels like this is a very specific workaround for the really more general problem of do we support post-install scripts (and user creation) for packages that require a reboot or that are installed by default
[2020-05-02 16:25] <PulkoMandy> nielx: only the ones installed by default, and there is only a single affected packages
[2020-05-02 16:26] <PulkoMandy> I think reboot is fine thanks to the activated-packages file?
[2020-05-02 16:26] <AGMS> It also applies to future packages that need a reboot.
[2020-05-02 16:26] <waddlesplash> no, it doesn't
[2020-05-02 16:26] <waddlesplash> those can have their users created before the reboot
[2020-05-02 16:26] <AGMS> They don't get their post-install run.  Though the users would get created.
[2020-05-02 16:27] <AGMS> (user creation done as part of the transaction, can roll it back too)
[2020-05-02 16:27] <AGMS> Also makes me wonder if packages mind if their users are created after or before they start running.
[2020-05-02 16:27] <nielx> so my suggestion would be: for beta 2 let's do it in a post-install script
[2020-05-02 16:28] <nielx> and move the ticket over to beta 3 for a proper fix
[2020-05-02 16:28] <PulkoMandy> yes
[2020-05-02 16:28] <AGMS> Sounds good, option 1 quick fix and do the better one later.
[2020-05-02 16:28] <nielx> I think the questions AGMS is asking are valid
[2020-05-02 16:28] <PulkoMandy> so the quickfix should be done in the beta2 branch directly, and not in master
[2020-05-02 16:28] <nielx> and do need an answer, especially if we want to support custom installs/images for R1
[2020-05-02 16:28] <AGMS> Oh, and can someone change the owner of the bug to AGMS instead of agmsmith?
[2020-05-02 16:28] <nielx> will do
[2020-05-02 16:29] <PulkoMandy> done
[2020-05-02 16:29] <nielx> thanks AGMS
[2020-05-02 16:29] <nielx> Moving on to https://dev.haiku-os.org/ticket/15808

[AGMS]

Did the quick post-install script fix to add the users there, which does get run on first boot. Pull request sent to HaikuPorts at ​https://github.com/haikuports/haikuports/pull/4945, and you can see it in my GitHub at ​https://github.com/agmsmith/haikuports/tree/OpenSSHPostInstallFix

[AGMS]

I tried adding a new BMessage command to PackageDaemon, but it turns out the actions for post-install processing are built into CommitTransactionHandler. Rather than duplicate code, the next step is to try to make a dummy transaction that installs all the packages as if they are all not installed. This should have the good side effect of writing the initial activated-packages file (currently if there is none, it doesn't get created until you install something).

[waddlesplash]

Change on HaikuPorts merged; but we should fix the underlying issue and then remove the script. Deferring to beta3.

[AGMS]

Patch at ​https://review.haiku-os.org/c/haiku/+/2342 is mostly working, but needs to fix a few niggles (package_daemon crash if actions done too early) to make it perfect. Will work on it as time permits (it's not needed until after R1Beta2).

[AGMS]

Updated version on Gerrit now is more internal to package_daemon, and seems to work.

[pulkomandy]

Patch submitted in hrev54986. Thanks!