wiki:Obsolete/MovedToTree/PackageManagement/OldIdeas

Context Navigation

Version 6 (modified by brecht, 15 years ago) ( diff )
reference correction

Requirements
Implementation
References

Package Management Ideas

This page is a place to hash out ideas regarding Haiku's package management (and creation). The following is a draft specification for the package management system to be included in R1. It is based on (1), (2) and the discussion in (5).

Requirements

This section describes the intended user experience.

Bundles

There seems to be a general preference for a bundles-based approach. Such a bundle is a single file that contains the application. This makes self-contained and easy to handle. The application can be run by double-clicking the bundle icon. Obvious exceptions to this rule are drivers and libraries. These have to be installed to be of any use. The contents of a bundle can be inspected by opening the bundle by means of a context-menu option.

Optionally, a bundle can be installed by moving it to /boot/apps (system-wide) or in /boot/home/<user>/apps (user-local). Another option is to right-click the icon and select "install for everyone" (only admins) or "install only for me". Any initial configuration (accepting a license) can be performed the first time a bundle is being run.

When a bundle has been installed, shortcuts to the application appear in the Deskbar menu. Applications in the Deskbar menu are subdivided into a number of a predefined categories (games, graphics, internet, ...).

brecht: I don't like Waldemar's idea of having bundles spread all around the filesystem, hence the clear distinction between installed and non-installed bundles. This might be a necessity for multi-user too.

Management

Uninstallation is performed by simply removing the bundle.

A user's application settings are kept when a bundle is uninstalled or deleted. The system however provides a comprehensive listing of applications for which user settings exist. The user can choose to delete settings for each of the bundles.

Application folders (/boot/apps and /boot/home/<user>/apps) display the list of installed bundles including information (description, availability of updates, security risk warning), just like the mail folder in BeOS displays emails.

Dependencies

The user does not have to worry about dependencies. If a bundle depends on other bundles, the system should make sure they are fulfilled (on run). If a dependency is not fulfilled, the system will ask the user for confirmation and automatically install any required bundles.

Updates

All installed software can be checked for new versions. For each of the installed applications, the changes with respect to the installed version can be displayed. Updates can be flagged as 'strongly recommended' when security problems are known. The user can select which applications he/she would like to update and have the system perform the updates.

Implementation

In this section the implementation of the system is discussed.

Bundles

A bundle is a plain zip archive that contains:

application executable(s) & data
metadata
- name
- version
- (revision?)
- (hash?)
- author
- homepage
- license
- category (for grouping applications in the Deskbar menu)
shortcuts to appear in the Deskbar menu
- a default shortcut to run when the bundle is being "run"
- right-clicking bundle could offer the option of opening a help document about the application

Run from Anywhere

On running a bundle, the archive's contents have to be accessed somehow. Either they can be extracted to a directory hidden from the user. While this can work well for smaller applications, larger applications (think commerial games) will occupy almost twice the size.

Another option is to read the individual contents from the zip archive as they are needed. Even if this is technically feasible, performance will suffer, especially if bundles are compressed heavily.

Some ports will need to know the absolute path to their data. Methods to solve this:

pkgfs (3)
assignfs (4)

With pkgfs, all ports are mounted under /boot/common.

With assignfs, each port is mounted in its own directory: /boot/apps/<port>-<version>-<revision>

Dependency Hell

Dependency hell is a problem mostly for ports. That does not mean it can be ignored. At least in the early years of Haiku, ports will be an important source of software.

Avoiding

Bundles are always fat bundles. All required libraries are included in the bundle. Problem solved! But, it feels like quitting. :)

Also, it is not a realistic solution for bundles that depend on Python or Perl for example.

Tackling

In order to solve conflicting dependencies, it is necessary to be able to have multiple versions of a library installed. Even worse, some libraries can be built with different options.

To make this work, it is obvious that a central bundle repository is required: HaikuBits. Alongside offering a browsable directory of software like BeBits, it stores information about dependencies. Dependency information (problems arising from certain combinations of bundles) is updated by the community.

An example. When ABC-1.0 is released, its dependency libfoo is at version 1.2.10. Bundle ABC-1.0 specifies "libfoo >= 1.2.10" as a dependency. Later, when libfoo 1.2.12 is released, it appears that this breaks ABC-1.0. HaikuBits is updated to indicate this: "libfoo >= 1.2.10 && != 1.2.12". When ABC-1.0 is now downloaded from HaikuBits, the bundle contains the updated information. A software updating tool can also check HaikuBits to see whether dependencies are still OK.

Because the act of porting can introduce additional incompatibilities, each port should be tagged with a revision number to uniquely identify it. Revisions can also be used to differentiate between ports with different build options. Specifying build options in the dependency information seems overkill anyway, as we should strive to have as few port revisions as possible (developers should have dependencies installed as bundles before porting).

The bundle metadata needs to be extended to include information about the dependencies:

minimum/maximum version
preferred version/revision
non-working versions/revisions

While bundles will not be available for download for retail software, it still makes sense to record dependency information about it on HaikuBits.

Having an application use a particular library version can be done by manipulating LD_LIBRARY_PATH or by virtually placing the library in the applications directory by means of assignfs or pkgfs.

libalpm

(and it's tool: pacman)

libalpm is the package management library used on ArchLinux, most people know it as "pacman" since that's the main tool to use, however, all the functionality is part of the libalpm library which could be utilized to create a nice GUI frontend for the package manager. It of course can also be adapted.

It uses libarchive to extract archives, and either libdownload or libfetch to download files - although one can also have it use an external command, like curl or wget.
The current status is this:
As far as libfetch is concerned: compiles and is linked to, but it doesn't really work, so I'm using curl instead - it works like a charm.
The important part: libarchive needed some work to support zip files in a useful way. Basically, it now supports seeking (which it didn't before), the central-directory headers for ZIP files (so it supports stuff like symlinks), and BeOS file attributes! Also, when reading from a source which doesn't allow seeking (... which are... none - on our case) it simply reads the local headers, but can also - if explicitly requested - provide "update"-entries to update the raw data when the central directory is reached (but those are of no importance anymore).

What's good about libalpm? Well, it provides useful configuring mechanisms, it stores dependencies and can also give you a list of which packages require a certain package. It keeps a database containing package information, including a file list. Configuration files in packages can be listed as such, which causes them to be installed as *.pacnew when they are upgraded (unless the new and old files equal - an md5 sum check is used there.) It provides the ability to use different database directories which allows us to have an automated way of creating package bundles. For instance, I can set the installdir to /tmp/mypackage and install the game "einstein" including its dependencies there, then move /tmp/mypackage/einstein/common/lib to /tmp/mypackage/einstein/lib, remove the unnecessary manpages, share files (well, usually anything else which is in the common/ folder), and then strip those dependencies from einstein's .PKGINFO file and create a bundled package which I can then install normally to say /boot/apps.

Another useful feature is the possibility to change the root directory. When a package contains a .INSTALL script, libalpm chroot()s into the root directory, cd()s into the installation directory, and then executes the .INSTALL script (which means, that install-scripts can and should work relative to the installation directory, although, if necessary, the absolute path is available in $PWD)

Also, libalpm works similar to an actual database. It doesn't blindly attempt to install a package, but first check for file conflicts, see if any files need backups or configuration files need to be installed as .pacnew, and then installs a package. If you install multiple packages at once, then it only either installs all of them, or none. It allows you to find the owning package of a file in the filesystem as well as listing all the files and dependencies of a package.

Where does it get the packages form? Two possibilities: One can use package files directly - which could be made in such a way that you could also just unzip them. In fact, it might be useful to put the .PKGINFO into the zip file as some extra data which is not unzipped when simply using `unzip´, although package creation is easier if it's just a file. The other one being repositories. The pacman utility currently allows you to list repositories like this in pacman.conf:

[core]
Server = file:///MyRepositories

[devel]
Server = file:///MyRepositories

[public]
Server = http://www.public-repository.com/

When you synchronize the repository databases, pacman downloads the file <Server>/<Reponame>.db.tar.gz which contains a list of packages with dependencies. When you install a package from such a repository, it downloads them from the very same location: <Server>/<Package File> The repo.db.tar.gz files are currently created using the tools repo-add and repo-remove provided in the pacman package. Those extract information from the .PKGINFO file and put it into the database which can be used as a repository then.

Here's a little log of using pacman to install a package file, and bundle a package with dependencies together into one package file. http://stud4.tuwien.ac.at/~e0725517/using-pacman-on-haiku.log.txt

Pros

It has been used on archlinux for a long time - so it works.
It's obviously possibly to compile and use it on haiku
Since most of its functionality is part of a library, it can be reused to build a GUI-application utilizing libalpm
pacman also provides scripts for building packages using a PKGBUILD script.

Cons

Your turn :P
(It's not BePorter, but hey, it might be possible to use them together: BePorter to create packages and bundles, and libalpm for managing the installed packages.)