HAIKU package management has broken the haiku file structure.

[bbjimmy]

60 to 80 percent of the third party software for Haiku has been broken by changes to the file system. The user guide describes the proper use of the /boot/common and /boot/home/config directory trees. These are now broken. Software that expects to add a binary or lib in either tree cannot. Scripting that expects binarys in /boot/home/config/bin or /boot/common/bin cannot find the binary in the /boot/home/config/non-packaged/bin directory.

The package management scheme has not kept to the design criteria:

Package management should be a convenient add-on to Haiku.

Package management must not change how a user deals with his system.

Package management must not break other software.

It would be nice to think that all software used on haiku will eventually be packaged in an .hpkg file. This is not now and will never be the case. Haiku needs the application developers. Breaking all their hard work is not a way to keep them interested in developing software for haiku.

To fix this:

/boot/home/config/bin and /boot/home/config/lib must not be read only.

return /boot/common and make it read/write.

Place packaged libs and binarys in /boot/home/config/packaged/bin and /boot/home/config/packaged/lib and these can be read-only.

When running a packaged file. ignore /boot/home/config/bin, /boot/home/config/lib, /boot/common/bin, and /boot/common/lib as these are not accounted for with the package management.

This will un-break all of the software that has been broken with package management and give the developers back the platform they developed their programs to run on. It will then make the .hpkg file a plus for the devs, not an un-do burden.

[Giova84]

Personally i have repackaged all old applications that i used before of Haiku PM (using the command "package create /path/to/package") and all these apps for me work properly again (also in /boot/home/config/bin and so on). If some old app still relies on /boot/common or /boot/apps and the source code is not available, this can be solved by making legacy links on Haiku or open these apps using DiskProbe and fix old paths.

[umccullough]

This will un-break all of the software that has been broken with package management and give the developers back the platform they developed their programs to run on.

Well, to be fair, anyone developing for Haiku has been gambling that it was going to change - that's sort of the nature of alpha software.

/boot/common was never a feature of BeOS, so it wasn't necessarily guaranteed to be a feature of Haiku either. The better process would probably be to repackage software as .hpkg so it would be able to take advantage of future package management features rather than just littering the filesystem with stuff in random locations when it's unzipped to /boot. The "unzip to /boot" mechanism was a serious hack - and nobody should have ever expected that to work forever. Besides being extremely messy, it was also a potential security nightmare for future Haiku users. Imagine someone who provided a malicious package that unzipped stuff to /boot/system or /boot/common, or even /boot/home/config that renders the system unbootable (which would be amazingly easy).

[bbjimmy]

Well, to be fair, anyone developing for Haiku has been gambling that it was going to change - that's sort of the nature of alpha software.

Not true. The devs promised that if it was documented, as /boot/config was, it would not change.

With the system as it is, does one need to package a binary to test it on haiku? if not, how will he get his binary in /boot/home/config/bin or a library in /boot/home/config/lib?

Yab requires me to place my yab library files, scripts really, in /boot/home/config/lib/yab to include them in a compiled executable. If I test the code before compiling it, the yab binary must be in /boot/home/config/bin. This not only breaks yab, it breaks the ability to develop yab software for Haiku.

[umccullough]

Replying to bbjimmy:

Not true. The devs promised that if it was documented, as /boot/config was, it would not change.

The only guarantee is that find_directory() would return the proper directory path based on the constant.

This is why zipfile distribution of packages with hardcoded directories is "broken by design", and any app that doesn't use the find_directory() constants to find system directories is also broken by design.

Furthermore, any app that requires libs to be put in a specific directory as you suggest is very much broken by design.

[bbjimmy]

I have seen this argument before, with Haiku devs complaining about up-stream projects not accepting simple and reasonable changes. Now, the shoe is on the other foot. The Haiku devs are the up-stream project and they just broke all the work of many many application devs, all doing their work in their spare time. I didn't write YAB or the YAB buildfactory, but these are the tools I need to develop my software for Haiku.

These changes to the directory tree, especially the read only status of ~/config have broken my ability to code. I work just as hard on my code as the Haiku devs, and in the long run for haiku, devs like me are just as important to the operating system as kernel devs. YAB is to haiku as Visual Basic was to Windows. A lot of software is written in it, or at least sketched out with it.

Zipfile distribution and symlink, "drop this file here" are not the only distribution systems that are broken. BeOS style .pkg files are broken as well.

What is wrong with moving packaged file directories to ~/config/packaged/bin and ~/config/packaged/lib and keeping them read-only? Or at least giving us a directory to place our bin files that will be picked up by the "#!/boot/home/config/bin/executable" - yab in my case - scripting line. As for the lib files I do not know of a work around, I may need to look into the buildfactory code, although I do not understand most of it.

Just because the application devs do not understand the system facilities as well as the core system devs does not mean their work is not important. Yes, we don't do everything "right", but we do add functionality to the operating system. A little help is all we need.

[bonefish]

Replying to bbjimmy:

I have seen this argument before, with Haiku devs complaining about up-stream projects not accepting simple and reasonable changes. Now, the shoe is on the other foot. The Haiku devs are the up-stream project and they just broke all the work of many many application devs, all doing their work in their spare time. I didn't write YAB or the YAB buildfactory, but these are the tools I need to develop my software for Haiku.

Zipfile distribution and symlink, "drop this file here" are not the only distribution systems that are broken.

Zip files that extract to self-contained directories (often used for applications) will continue to work. Zip files that hard-code directory paths -- e.g. by requiring unzipping at some particular place or including "drop here" symlinks -- have not been future-compatible in the first place. Now is the time they break. Simple repackaging would "fix" most of them, BTW.

BeOS style .pkg files are broken as well.

#8495 and #10004 track the PackageInstaller issues.

What is wrong with moving packaged file directories to ~/config/packaged/bin and ~/config/packaged/lib and keeping them read-only?

The goal is that eventually virtually all software will be packaged. Therefore most users would have an essentially empty top-level directory and a "packaged" subdirectory that would contain everything. Since that's a bit silly, the decision has been made to do it the other way around.

Or at least giving us a directory to place our bin files that will be picked up by the "#!/boot/home/config/bin/executable" - yab in my case - scripting line.

Just don't use an absolute shebang. On Haiku you can use "#!executable" to have PATH be searched for the executable.

[scottmc]

The only guarantee is that find_directory() would return the proper directory path based on the constant.

Well, it seems B_COMMON_* have been removed, so find_directory() don't help in that case... that's probably the root cause of #10099, which I'm looking into now.

[umccullough]

Replying to scottmc:

Well, it seems B_COMMON_* have been removed, so find_directory() don't help in that case... that's probably the root cause of #10099, which I'm looking into now.

By "removed", you mean deprecated and removed from the public header so that source code will be forced to be updated.

find_directory.cpp still defines the "missing" constants, so that existing apps will still work. The ordinal values of those constants is preserved, and skipped over in the FindDirectory.h to make sure they're not re-used.

So yes, find_directory() will still be guaranteed to work with these BeOS-derived constants, but newly compiled software will need to be tweaked to no longer use them.

[bbjimmy]

I worked myself up to get really pissed off about this stuff, I mean, after all, I have been programming on personal computers since before there were personal computers. I made my first one with a four bit microprocessor and did all the bios stuff myself. I have been programming since before most of the "kids" around here. but then I realized that I am getting to the point that I do not like change. This change to the file system seems silly and un-necessary to me. Then.... umccullough talks about find_directory(), a c++ call, I am using BASIC, not c or c++, I never liked c, even when working with drivers for dec vax, and I never got my head to wrap arround the idea of objects. and bonefish adds the #!executable, I never heard of that.

I found the finddir command line tool that works for BASIC scripts and yes indeed, #!yab finds the interpreter. Still, it makse a lot of work for application devs to change their stuff just to keep up with disruptive changes. I don't like to have to go back and fix stuff that wasn't broken.

[axeld]

I don't like to have to go back and fix stuff that wasn't broken.

Well, they were broken, they just didn't fail yet as the file structure only saw some slight changes over the past years. Those also could have broken applications before that relied, for example, on /boot/beos; /boot/common has been introduced with Haiku. Be made it clear from the start that you must not depend on fixed paths, but that you should instead use the find_directory() API. That yab does not do this is its own fault, and that's the fault your suffering from.

Nobody likes change when there is no reason to change things. But package management is a huge addition, and, in order to keep things clean, warrants a deep impact on the overall system structure.

Of course it sucks, that it renders Haikuware pretty much useless. But that it does so now, doesn't mean there won't be a solution to that problem before the next release. Any zip installation that uses fixes paths is broken by design, but that doesn't mean it cannot be fixed in a way that doesn't annoy users too much.

Package management is the future of Haiku software distribution; if Haikuware won't embrace that, it will go away.

[X512]

My opinion on this problem: why not to merge packages with "non-packaged" in "/boot/home/config" folder? It solve all problems with compatability without hacks. If someone will try to write to "/boot/home/config" it will go to "non-packaged".

Also I don't like idea of removing "/boot/common", because it break safe mode. Unstable drivers can make Haiku unbootable if installed in "/boot/system".

[pulkomandy]

X512: it is not easy to detect where a write should go. There already are shine-through directories (system/var for example), some packages have writable files, as well. This also adds an extra place to look when reading the directory, the result is slower access to the disk.

With the current solution, access to the packages *replaces* access to the on-disk FS, and it is faster. If we still have to access the disk, access to the packages *adds* time to the on-disk access, and the system will feel slower.

As for removal of common, you could still boot without "non-packaged", or even boot with the minimal dependencies of the "haiku" package to get some kind of safe mode.