[packagefs] Cannot handle package activations with identical names

[waddlesplash]

If you try to update or activate a package with an identical name (including revision) to an installed package, this occurs:

[system] Applying changes ...
*** failed to commit transaction: Failed to change the package activation in packagefs: Name in use

That's because of ​this code in packagefs.

Probably solving the TODO about looking packages up by node_ref rather than name would fix this.

[diver]

Would be nice to fix before beta2 as it might prevent system updates.

[pulkomandy]

System updates always come with a different version or revision number so they are not affected. Only trying to reinstall an already installed package will fail.

[diver]

Still this is somehow happens: after a fresh install pkgman tries to update a selection of packages with the ones with the same name.

[waddlesplash]

IIRC it was hard to get this problem to reproduce in the past; so are you saying this always reproduces under a beta2 image?

[diver]

I haven't tried beta2 images yet but I could reproduce it easily using nightlies after a fresh install.

[waddlesplash]

OK, then we should at least investigate why this occurs and fix that bug. It is indeed extremely odd.

[Forza]

Hello,

This problem happens with the beta2 release-candidate too (haiku-r1beta2-hrev54154_107-x86_64-anyboot.zip) Currently, every time I make a clean install and run the sw updater.

​https://paste.tnonline.net/files/vNna30rJuAt6_haiku-update-fails-on-clean-install.png

[waddlesplash]

Can you please run "pkgman full-sync" in a Terminal, and paste the output of what it says it is going to do here?

[waddlesplash]

Ticket retargeted after milestone closed

[nielx]

Replying to Forza:

Hello,

This problem happens with the beta2 release-candidate too (haiku-r1beta2-hrev54154_107-x86_64-anyboot.zip) Currently, every time I make a clean install and run the sw updater.

​https://paste.tnonline.net/files/vNna30rJuAt6_haiku-update-fails-on-clean-install.png

I think you were running into #16092 instead. Installer did not properly do a clean install and makes a mess of the package directory.

[mmu_man]

Well this always happens when I full-sync after pushing a recipe to HaikuPorts with a package I built and install myself for testing. It tries to replace the one I built (with me as packager) with the one from HP, which has the exact same version. So it's quite easy to reproduce : build any package yourself and install it, then try to full-sync.

[mmu_man]

Here it does this:

The following changes will be made:
  in system:
    upgrade package ocp_x86-0.2.2-1 to 0.2.2-1 from repository HaikuPorts
Continue? [yes/no] (yes) : 
100% ocp_x86-0.2.2-1-x86_gcc2.hpkg [4.53 Mio]
Validating checksum for https://eu.hpkg.haiku-os.org/haikuports/master/x86_gcc2/current/packages/ocp_x86-0.2.2-1-x86_gcc2.hpkg...done.
[system] Applying changes ...
*** failed to commit transaction: Failed to change the package activation in packagefs: Name in use

[mmu_man]

Admittedly it shouldn't happen with non-dev users who don't build their own packages.

[pulkomandy]

I suspect this could be related to the problem in haikuports, where if you fix a requires or provide and rebuild a recipe, it will fail to activate in the chroot?

[pulkomandy]

Bumping priority because this makes using haikuports annoying.

I guess packagefs is right to complain that another package with the same name is activated, but should notice that it is going to be replaced?

I guess one option is to make sure the "change activation" request sent to packagefs has the disabling of the old package first, and enabling of the new one only after that? Or it should be a "reactivate package" instead of an "activate package"? In which case the linked code for packagefs would be ok, and the problem would be in the package daemon instead. And indeed the daemon does activation first, then deactivation, and never uses the "reactivate package" code.

[kallisti5]

"the problem in haikuports"

aka endless loop of:

waiting for build package XXX to be activated
waiting for build package XXX to be activated
waiting for build package XXX to be activated
waiting for build package XXX to be activated
waiting for build package XXX to be activated

Just mentioning it here to make the error searchable :-)

[stippi]

I've run into this, and think I am providing a correct fix with this changeset:

​https://review.haiku-os.org/c/haiku/+/3597

Please have a look.

[stippi]

Replying to pulkomandy:

I guess packagefs is right to complain that another package with the same name is activated, but should notice that it is going to be replaced?

The "Name in use" error, as far as I can tell, is from the FS layer. So this shouldn't be about updating a package with the "same name", the issue is replacing a package file with an identical copy from a new transaction.

I guess one option is to make sure the "change activation" request sent to packagefs has the disabling of the old package first, and enabling of the new one only after that? Or it should be a "reactivate package" instead of an "activate package"? In which case the linked code for packagefs would be ok, and the problem would be in the package daemon instead. And indeed the daemon does activation first, then deactivation, and never uses the "reactivate package" code.

I am not sure this has anything to do with packagefs. The error I saw is from the package server, which is responsible for committing one transaction at a time only (among other things). I think there may be other situations why a package file exists already in the packages folder, but one situation how this can form is when there are two concurrent transactions started, both of which depend on the same missing package. So it will be part of both transactions, one of them will be committed first, so the package file will be present at the time the second transactions is committed.

[pulkomandy]

Yes, now that I see your change proposal it makes sense, and it's a much simpler explanation. Looking at problems from another perspective seems to help :)

Thanks for investigating this and finding the actual problem!

[stippi]

Looking at the ticket description, the original error seems to have to do with packagefs (at the linked code). There seem to be two separate problems, I have fixed only one of them. What I fixed was during commiting the transaction.

[stippi]

I mean, both problems would be during committing a transaction... but I don't remember my error to mention packagefs.

[stippi]

One package I was able to reproduce this with (my version of the problem) was VLC, with pretty much nothing else installed. If memory serves.

[pulkomandy]

The change by stippi has been merged. The remaining part of this is a regression caused by hrev51233, which attempts to overwrite a package without removing it first. That approach is not correct, so I think hrev51233 should be reverted and the initial problem from #10038 fixed in a different way, probably by having the package server synchronize better with packagefs and do not try to resolve dependencies while a transaction is being applied?

[pulkomandy]

I cannot reproduce the original issue nor the haikuports one. Is it ok to fix this after stippi's fixes, or are there remaining problems?

[waddlesplash]

Can you reproduce it by copying a file out of /system/packages (preserving it as exactly named there) and then trying to "pkgman install" that file?

[nielx]

Another bump: is this issue still reproducible on recent nightlies?

[waddlesplash]

packagefs was never fixed to solve the underlying problem, so while pkgman may not trigger the bug most of the time, it is still definitely there.

[pulkomandy]

Move unsolved issues to beta 4 since there is no chance of fixing them now.

[pulkomandy]

I think with the pkgman side of things now fixed, it's ok to move this out of the beta milestone now? Even if the issue is not fully fixed, it is now harder to trigger in normal usage, making it less of a priority.

[waddlesplash]

Indeed it's hard to trigger in normal usage. I'm not sure it's fixed, at least the TODO in packagefs could be solved, but we can indeed bump this out of the beta.

[bipolar]

Just hit this one while trying to pkgman full my 32 bits beta4 installation. Not sure which package is causing the issue (I had LOTS of packages to update/sync, most from my local builds and testing of *_python3* packages).

Edit: got around it by removing from /system/packages/ 15 .hpkg files I locally built/installed, then pkgman full finally worked.