Crash when trying to start command-line applications and standard output or standard error stream is redirected

[monni]

hrev58436, but this has happened for last few days

Syslog:

KERN: debug_server: Thread 1034 entered the debugger: Segment violation
KERN: stack trace, current PC 0x12eaacc6ff7  </boot/system/lib/libroot.so> _ZN8BPrivate9hoardSbrkEl + 0x97:
KERN:   (0x7fac4d5d8f20)  0x12eaacc8193  </boot/system/lib/libroot.so> _ZN8BPrivate9hoardHeap9freeBlockERPNS_5blockERPNS_10superblockEiPNS_11processHeapE + 0x113
KERN:   (0x7fac4d5d8f50)  0x12eaacc92da  </boot/system/lib/libroot.so> _ZN8BPrivate11processHeap7getHeapEi.part.0 + 0x2a (closest symbol)
KERN:   (0x7fac4d5d8f80)  0x12eaacc3d0a  </boot/system/lib/libroot.so> _ZL7do_execPKcPKPcS3_b + 0x1fa
KERN:   (0x7fac4d5d8fc0)  0x12eaacb7d55  </boot/system/lib/libroot.so> __wcstoul_internal + 0x165
KERN:   (0x7fac4d5d8fe0)  0x12eaac399e6  </boot/system/lib/libroot.so> _GLOBAL__sub_I_eh_alloc.cc + 0xa6
KERN:   (0x7fac4d5d91f0)  0x19b2a53ce4a  </boot/system/runtime_loader> _ZL17init_dependenciesP7image_tb.constprop.0 + 0x14a
KERN:   (0x7fac4d5d92a0)  0x19b2a53d82c  </boot/system/runtime_loader> load_program + 0x1dc
KERN:   (0x7fac4d5d9330)  0x19b2a544eb3  </boot/system/runtime_loader> runtime_loader + 0xf3

[waddlesplash]

Can't seem to reproduce here. I ran ls -alh 1>out.txt and it worked fine.

[monni]

I got consistent crash when trying to build anything... It either crashed when mounting empty hpkg, running diff or grep in configure script. Haiku was running less than 10 minutes when it crashed and if it didn't crash, it did print "Invalid Argument" in stderr. What it did say before the "Invalid Argument" changed depending on exact hrev, but it did often exit with signal 21 (SIGKILLTHR).

What I noticed was that when it crashed, it was using almost all of swap (48 kB remaining out of 16 GB), but only little over 1 GB of physical memory, of which about 600-800 MB was used by kernel.

[waddlesplash]

Where is all the memory being used? That sounds very strange. Figuring out when this started would be useful, too.

[monni]

I think there might be multiple issues, but I will try to figure out what hrev the crashing started, it shouldn't be hard as I know which hrev just print the error without crashing runtime_loader. It might take a few hours, but I don't have to go anywhere today...

First one that crash hard is hrev58421. Terminal seems to break in hrev58295, the exact error that revision gives is cannot make pipe for command substitution: Invalid Argument. There might be changes to the relevant files in later revisions, that would mean there is a regression somewhere.

[monni]

Applying ​https://gist.github.com/mtl1979/0ec0dc611409b9aaa1fcf1fdd6cb30de fixes both the crash and redirecting issue for me...

The root issue seems to be package_daemon loading incompatible libroot.so from haiku-hrev1~beta5_hrev57937_122-1-x86_64.hpkg when building new packages.

This means we can only add new syscalls to bottom of the list, so existing ones don't get reordered and no new parameters can be added to existing syscalls, unless the kernel-side code can ignore invalid values.

[waddlesplash]

As discussed on IRC, mixing differing libroot and kernel versions is not supported.

[monni]

I wasn't trying to mix different versions... The real issue is that as a developer I need to have headers and development libraries from beta 5 on a later revision (it doesn't matter if it is nightly or not). As such having different versions of libroot.so in /system/lib and /system/develop/lib is required. As pointed out by others on IRC, only working solution currently is having own partition for every hrev after hrev57937+113, which is used by buildmasters.

"haiku_devel" package must not depend on package "haiku" for anything in Haiku to be useful.

[waddlesplash]

Mixing and matching headers and libraries from beta5 with nightlies is explicitly not supported. And the haiku_devel package contains alternate versions of libroot and other files that can only work with the exact version of the base package.

The buildmasters use kernels compatible with beta5. If you need to use beta5 for building, then you need a beta5 kernel, or at least a nightly kernel of the same vintage.

[monni]

Not supporting using older headers and development libraries pretty much hinders using buildmasters for HaikuPorts useless. Other people use different "unsupported" methods or working around the limitations, but it seriously would be better to just support having "own" copy of "haiku_devel" inside chroot of HaikuPorter.

Even without mising bets 5 and nightlies, there is incompatibilities in headers between +113 and +122.

[waddlesplash]

You can put a haiku...hpkg and haiku_devel...hpkg in HaikuPorts' packages directory and HaikuPorter will pick that up when running builds. But of course this will only work properly if the libroot in the package is compatible with the running kernel.