Context Navigation

← Previous Change
Wiki History
Next Change →

Infrastructure

Timestamp:: Nov 27, 2013, 3:33:41 PM (11 years ago)
Author:: bonefish
Comment:: Update wrt. to removed /boot/common, up to and including section "Installation Location Order and Consistency".

Legend:

: Unmodified
: Added
: Removed
: Modified

Obsolete/MovedToTree/PackageManagement/Infrastructure

-              v7
+              v8
  * The '''package kit''' is an API for package management related programming.
  * The '''dependency solver''' is a part of the package kit. It solves dependencies between packages.
  * The '''package management daemon''' is a background process that is activated whenever the user adds packages to or removes them from one of their activation locations. It verifies that all dependencies are fulfilled (prompting the user, if necessary) and performs whatever package pre-activation/post-deactivation tasks are required. The daemon is also contacted by the package manager (after it has resolved dependencies and downloaded all required packages) to do the package de-/activation.
  * The '''package manager''' provides the user interface for software installation, update, and removal. There are actually two programs, `pkgman`, a command line tool, and a GUI application.
+ * The '''package management daemon''' (short: package daemon) is a background process that is activated whenever the user adds packages to or removes them from one of their activation locations. It verifies that all dependencies are fulfilled (prompting the user, if necessary) and performs whatever package pre-activation/post-deactivation tasks are required. The daemon is also contacted by the package manager (after it has resolved dependencies and downloaded all required packages) to do the package de-/activation.
+ * The '''package manager''' provides the user interface for software installation, update, and removal. There are actually two programs, `pkgman`, a command line tool, and, `HaikuDepot`, a GUI application.
 [[Image(package-management-infrastructure.png, align=center)]]
 …
 == Software Installation Locations ==
 In Haiku there are three places where software is installed. "/boot/system", "/boot/common", and "/boot/home/config". "/boot/system" is the place where the base system is installed. It is immutable and should not be affected by third party software. Only installing a new system release/update will change anything there. "/boot/common" is the place where third party software can be installed system-wide (i.e. for all users), while "/boot/home/config" is only for the user's software. The discrimination between those two doesn't make that much sense yet, but when Haiku supports multi-user it will (obviously each user needs their own home directory then, e.g. "/boot/home/<user>").
+In Haiku there are two main locations where software is installed. "/boot/system" and "/boot/home/config". "/boot/system" is where system-wide software (i.e. software for all users), including the base system, is installed, while "/boot/home/config" is only for the user's software. The discrimination between those two doesn't make that much sense yet, but when Haiku supports multi-user it will (obviously each user will need their own home directory then, e.g. "/boot/home/<user>").
 There are also the directories "/boot/apps" and "/boot/preferences" which we inherited from BeOS. With the migration to package management those will become symlinks to "/boot/common/apps" and "/boot/common/preferences" respectively, or we might even remove them completely.
+At both main installation locations a packagefs instance is mounted. Each instance presents a virtually extracted union of the contents of all packages in the subdirectory "packages" of that location. E.g. if one would extract the contents of all package files in "/boot/system/packages" to an actual directory, that would match exactly what is presented by packagefs in "/boot/system". With a few exceptions -- packagefs provides several additional directories.
+At each of the three installation locations a packagefs instance is mounted. Each instance presents a virtually extracted union of the contents of all packages in the subdirectory "packages" of that location. E.g. if one would extract the contents of all package files in "/boot/system/packages" to an actual directory, that would match exactly what is presented by packagefs in "/boot/system". With a few exceptions -- packagefs provides several additional directories.
+There are so-called shine-through directories which live on the underlying BFS volume. Normally a file system mounted at a directory would completely hide anything that is in that directory. These shine-through directories are handled specially, though; packagefs shows them just like they are on the BFS volume. One of those directories is "packages". This is necessary since otherwise it wouldn't be possible to add, remove, or update any packages. In "/boot/common" and "/boot/home/config" there are a few more shine-through directories like "settings", "cache", and "non-packaged". The latter is a place where software can be installed the "traditional", i.e. unpackaged, way.
+There are so-called shine-through directories which live on the underlying BFS volume. Normally a file system mounted at a directory would completely hide anything that is in that directory. These shine-through directories are handled specially, though; packagefs shows them just like they are on the BFS volume. One of those directories is "packages". This is necessary since otherwise it wouldn't be possible to add, remove, or update any packages. Further shine-through directories are "settings", "cache", "var", and "non-packaged". The latter is a place where software can be installed the "traditional", i.e. unpackaged, way.
 …
 === Manual Installation ===
 At the lowest level software can be installed by simply dropping a respective package file in a "packages" subdirectory of one of "/boot/system", "/boot/common", or "/boot/home/config". packagefs monitors the directory and, when happy with the newly added package, just presents its contents on the fly in the directory structure. The package is said to be activated. Removing a package has the opposite effect.
+At the lowest level software can be installed by simply dropping a respective package file in a "packages" subdirectory of one of "/boot/system" or "/boot/home/config". The package daemon, a background process that sleeps most of the time, monitors the directory and, when happy with the newly added package, it asks packagefs to presents its contents on the fly in the directory structure. The package is said to be activated. Removing a package has the opposite effect.
 Things are a bit more complicated due to the fact that packages usually have dependencies. E.g. when adding a package that has an unsatisfied dependency (e.g. needs a certain library that is not installed) it is not a good idea to activate the package nonetheless. The package contents (programs, libraries,...) might not work correctly, and, e.g. when shadowing other installed software, might even break things that worked before.
 The solution is that packagefs doesn't blindly act on its own when packages change. Instead the package management daemon, a background process that sleeps most of the time, examines the new situation and checks whether all dependencies are fulfilled. If they are, it tells packagefs to activate/deactivate the packages as requested. In case there are issues with the dependencies, according to how it has been configured via settings, the daemon prompts the user immediately, checks remote repositories for solutions to the problem and presents the user with the possible options, or it even performs all necessary actions without bothering the user, if possible. In the end, if the problems could be solved (e.g. by downloading additional packages), the respective packages will be de-/activated, or, otherwise, nothing will be changed.
+That's why the package daemon doesn't just activate any well-formed packages. Instead it examines the new situation and checks whether all dependencies are fulfilled and whether there are any conflicts. If they aren't any problems, it tells packagefs to activate/deactivate the packages as requested. In case there are issues with the dependencies, according to how it has been configured via settings, the daemon prompts the user immediately, checks remote repositories for solutions to the problem and presents the user with the possible options, or it even performs all necessary actions without bothering the user, if possible. In the end, if the problems could be solved (e.g. by downloading additional packages), the respective packages will be de-/activated, or, otherwise, nothing will be changed.
 To avoid always having to check all dependencies when booting, the package daemon writes the last consistent state of package activations to the file "packages/administrative/activated-packages". When being mounted packagefs, reads that file and only activates the packages specified by it. If the file is missing or packages it refers to cannot be found or loaded, packagefs falls back to activating all packages in the "packages" directory. The package daemon, once started, checks the state.
 …
 === Installation Location Order and Consistency ===
 Having three separate installation locations for software requires some considerations regarding their consistency and interactions. There's a well-defined order of the installation locations: "/boot/home/config", "/boot/common", "/boot/system". This has already been the order in which on BeOS commands, libraries, and add-ons where searched (according to the environmental variables `PATH`, `LIBRARY_PATH`, and `ADDON_PATH`; with the exception that "/boot/common" was only planned but didn't exist yet). That e.g. allows a user to install a new/different version of a program in "/boot/home/config" and have it override the version in "/boot/common".
+Having two separate installation locations for software requires some considerations regarding their consistency and interactions. There's a well-defined order of the installation locations: "/boot/home/config", "/boot/system". This has already been the order in which on BeOS commands, libraries, and add-ons where searched (according to the environmental variables `PATH`, `LIBRARY_PATH`, and `ADDON_PATH`). That e.g. allows a user to install a new/different version of a program in "/boot/home/config" and have it override the version in "/boot/system".
 This order also needs to be the order in which package dependencies are directed. While it shall be possible to have a library in "/boot/home/config" override one in "/boot/common" and have programs installed in the latter location use the overriding library, packages in an installation location must not have dependencies that can only be resolved to packages installed in a location that is prior according the order. E.g. a program installed in "/boot/common" must not depend on a library that is only installed in "/boot/home/config". When going multi-user that would mean the program would work for one user, but not for another one who hasn't installed the library. Similarly "/boot/system" is the fully self-contained base system. All dependencies must be resolved within it. A safe-mode boot should be possible with only the "/boot/system" packagefs being mounted. As a consequence these constraints have to be respected when software is installed or uninstalled.
+This order also needs to be the order in which package dependencies are directed. While it shall be possible to have a library in "/boot/home/config" override one in "/boot/system" and have programs installed in the latter location use the overriding library, packages in an installation location must not have dependencies that can only be resolved to packages installed in a location that is prior according to the order. E.g. a program installed in "/boot/system" must not depend on a library that is only installed in "/boot/home/config". When going multi-user that would mean the program would work for one user, but not for another one who hasn't installed the library. Consequently "/boot/system" is fully self-contained. All dependencies must be resolved within it. A safe-mode boot should be possible with only the "/boot/system" packagefs being mounted. As a consequence these constraints have to be respected when software is installed or uninstalled.
 Another challenge that comes with having three installation locations is that some packages have compiled-in absolute paths to their own files (e.g. data files) or to their dependencies. The former could be solved by building three different versions of a package, but that wouldn't work with the latter and would be tedious anyway. The solution are dynamically generated symbolic links in a fixed location, "/boot/system/package-links", that for each installed package and its dependencies refer to the respective installation location.
+Another challenge that comes with having two installation locations is that some packages have compiled-in absolute paths to their own files (e.g. data files) or to their dependencies. The former could be solved by building two different versions of a package, but that wouldn't work with the latter and would be tedious anyway. The solution are dynamically generated symbolic links in a fixed location, "/boot/system/package-links" (symlinked to "/packages"), that for each installed package and its dependencies refer to the respective installation location.
 For each installed package a subdirectory named like the package (package name plus version) will be generated automatically. That subdirectory contains a symlink ".self" which refers to the installation location of the package itself as well as a symlink for each of its dependencies pointing to their installation locations. E.g. for an OpenSSH package installed in "/boot/common" and OpenSSL installed in "/boot/system" the directory could look like this:
+For each installed package a subdirectory named like the package (package name plus version) will be generated automatically. That subdirectory contains a symlink ".self" which refers to the installation location of the package itself as well as a symlink for each of its dependencies pointing to their installation locations. Furthermore there's a symlink ".settings" which points to the installation location's directory for global settings. E.g. for an OpenSSH package installed in "/boot/home/config" and OpenSSL installed in "/boot/system" the directory could look like this:
 {{{
 /boot/system/package-links/openssh-5.8p2-1/
+        .self           -> ../../common
+        .self           -> ../../../home/config
+        .settings       -> ../../../home/config/settings/global
         haiku           -> ../..
         lib:libssl      -> ../..
 …
 While it depends on the location the package has been installed in where the paths refer to, the absolute paths of the package links themselves remain stable. So they can be compiled in, when a package is built, and will work regardless of where the package is installed.
 Another problem the package links can solve are incompatible versions of the same package being installed in different locations. E.g. when a program and another program it depends on are installed in "/boot/common", installing an incompatible version of the latter in "/boot/home/config" will not break the former, since the dependency link will continue to point to the compatible version. With a bit of help from the runtime loader the same would work for libraries. In practice that's less of a problem, though, since libraries usually have a naming scheme and matching shared object names that already prevent mismatches.
+Another problem the package links can solve are incompatible versions of the same package being installed in different locations. E.g. when a program and another program it depends on are installed in "/boot/system", installing an incompatible version of the latter in "/boot/home/config" will not break the former, since the dependency link will continue to point to the compatible version. With a bit of help from the runtime loader the same would work for libraries. In practice that's less of a problem, though, since libraries usually have a naming scheme and matching shared object names that already prevent mismatches.