[runtime_loader] can't find libgcc_s.so.1 when LD_PRELOAD is used

[diver]

hrev49408.

The following command used to work without errors:

LD_PRELOAD=/system/lib/x86/libroot_debug.so /system/apps/WebPositive http://depot.haiku-os.org
runtime_loader: Cannot open file libgcc_s.so.1: No such file or directory

LD_PRELOAD="/boot/system/lib/x86/libgcc_s.so.1 /system/lib/x86/libroot_debug.so" /system/apps/WebPositive http://depot.haiku-os.org
runtime_loader: /boot/system/lib/x86/libgcc_s.so.1: Could not resolve symbol 'pthread_cancel'
resolve symbol "pthread_cancel" returned: -2147478780
runtime_loader: /boot/system/lib/x86/libgcc_s.so.1: Troubles relocating: Symbol not found
runtime_loader: Cannot open file libgcc_s.so.1: No such file or directory

[pulkomandy]

Possibly related to the recent update of the gcc package?

[waddlesplash]

Not sure. I had the same issue a few days ago, before the GCC package update, IIRC.

[korli]

 LD_PRELOAD=/system/lib/x86/libroot.so /system/apps/WebPositive displays the same message about libgcc_s.so.1.

[bonefish]

readelf -W --syms .../libroot_debug.so anyone?

[korli]

output of readelf -W --syms /system/lib/x86/libroot_debug.so

[bonefish]

pthread_cancel is properly exported at least (also no symbol versioning used). Same info for libgcc_s.so.1, please. Also an strace might help (with "-l").

[korli]

output of readelf -W --syms /system/lib/x86/libgcc_s.so.1

[korli]

LDPRELOAD=/system/lib/x86/libroot_debug.so strace -l -o WebPositive.strace.txt WebPositive

[bonefish]

The libgcc_s.so.1 symbols are as expected. The strace output is unfortunately incomplete. I played around a little with an older Haiku image I have at hand and often the beginning of the output is missing. So apparently something is broken. :-/

Anyway, most of the loading phase still seems to be captured. The images have been loaded and relocated at line 572. Thereafter the initialization code of the images is invoked -- libroot, libbe (lots of locale related stuff), libmedia, and finally WebPositive itself, around line 1243. A few lines later the BApplication is constructed.

So we have already entered main(). The problem must occur when loading some add-on/library dynamically. Since I see no mention of libgcc_s.so.1, this does apparently happen in another thread. An strace run tracing all threads would be helpful ("-T"; "-l" won't be needed).

[korli]

LDPRELOAD=/system/lib/x86/libroot_debug.so strace -T -o WebPositive.strace.txt WebPositive

[bonefish]

Thanks, korli!

I see no indication of the error or the output quoted in the ticket description being produced (no _kern_write(0x2,...) in the strace log. The "-t" strace option would also trace new teams spawned by WebPositive, which is where I suppose the error originates from. CC'ing the folks who might actually know what is supposed to happen at least.

[pulkomandy]

Could it be that the spawned team (I don't see what it could be) is a gcc2 application? Can't find the ticket right now, but there is a similar problem when trying to take a screenshot of Web+ with LD_PRELOAD forcing an x86 lib, because screenshot is a gcc2 application.

[bonefish]

Sounds plausible. The question is: Why has that changed (recently) respectively why is WebPositive requiring a gcc 2 application at start-up now?

[waddlesplash]

I don't think this is the case at all, because I was getting similar problems (except runtime_loader shows "can't find libstdc++" IIRC) trying to use LD_PRELOAD on a libbe I'd just compiled to run StyledEdit. I'm not at my dev machine ATM, but I will be in a few hours, so I'll try to reproduce the issue that way and upload strace output.

[diver]

Found the problem. Ot was introduced in hrev47677 with the merge of 'gcc_syslibs' branch.

[diver]

Perhaps the problem might be somewhere here? ​http://cgit.haiku-os.org/haiku/tree/build/jam/SystemLibraryRules#n166

[pulkomandy]

You can use git bisect to further bisect the commit inside the branch. It requires compiling Haiku yourself.

git bisect start hrev47677 hrev47676

git will checkout a commit in the middle of the range (including the branch). Build and test, then use either git bisect good or git bisect bad (or git bisect skipif that build fails for another reason). Until git tells you the exact commit creating the problem.

[korli]

An idea: When loading an x86 libroot_debug.so on a x86_gcc2 system, ​open_executable is used to load libroot_debug dependencies aka libgcc_s, but sSearchPathSubDir is still NULL because the application isn't loaded ​yet. libgcc_s isn't located in standard search paths for x86_gcc2 and cannot be found hence the failure message. It has maybe worked earlier when libroot_debug.so had no dependencies. In the end, it may be worthwhile to set sSearchPathSubDir also for libraries when not already set.

[diver]

Interestingly, copying the following libs:

/system/lib/x86/libgcc_s.so.1
/system/lib/x86/libroot_debug.so

into a lib folder next to an examined binary and invoking LD_PRELOAD=lib/libroot_debug.so mpv-x86 seem to work. However, this doesn't seems to work for Web+ as I get:

runtime_loader: /boot/system/lib/x86/libsupc++.so: Could not resolve symbol '___tls_get_addr'
resolve symbol "___tls_get_addr" returned: -2147478780
runtime_loader: /boot/system/lib/x86/libsupc++.so: Troubles relocating: Symbol not found

[mmu_man]

FWIW, I just had this same error when porting MuseScore. Somehow the build system ended up passing -Wl,-rpath,/boot/system/lib -Wl,-rpath,/boot/system/lib/x86 for a gcc4 build on a gcc2 hybrid. Setting CMAKE_SKIP_RPATH made it skip the first rpath, and it worked that way.

[waddlesplash]

Fixed in hrev51600, using more-or-less the model that korli proposed.

[pulkomandy]

Not sure this is right. This path is used for lookup of the "lib" dir next to the application. Now it will look for it next to libroot_debug instead (or whatever you preloaded first).

Maybe something like this would make more sense:

if (sSearchPathSubDir == NULL || type == B_APP_IMAGE)

That is, set it with anything we can grab as soon as we can, but then replace it with the app path when we get to loading that.

[waddlesplash]

If you look in the code below the changed, you will see that sSearchPathSubDir is set to either "x86_gcc2" or "x86", not a full path. So adding that second case will only matter if we load things from different ABIs in the same binary...?