arm linker relocation issues

[kallisti5]

Our gcc toolchain's linker (ld) continues to have problems with odd relocation issues.

We (waddlesplash) was able to work around these issues by leveraging the clang linker (lld) to link the operating system.

While the clang linker works as expected, the gcc linker throws relocation errors linking libbe:

kDir1 objects/haiku/arm/release/system/glue/arch 
MkDir1 objects/haiku/arm/release/system/glue/arch/arm 
As objects/haiku/arm/release/system/glue/arch/arm/crti.o 
Cc objects/haiku/arm/release/system/glue/init_term_dyn.o 
As objects/haiku/arm/release/system/glue/arch/arm/crtn.o 
Link objects/haiku/arm/release/system/libroot/libroot.so 
SetType1 objects/haiku/arm/release/system/libroot/libroot.so 
MimeSet1 objects/haiku/arm/release/system/libroot/libroot.so 
SetVersion1 objects/haiku/arm/release/system/libroot/libroot.so 
Chmod1 objects/haiku/arm/release/system/libroot/libroot.so 
Link objects/haiku/arm/release/kits/libbe.so 
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/ld: objects/haiku/arm/release/kits/interface/interface_kit.o: relocation R_ARM_REL32 against external or undefined symbol `_ZTIi' can not be used when making a shared object; recompile with -fPIC
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/ld: objects/haiku/arm/release/kits/interface/interface_kit.o(.ARM.extab+0x445c): unresolvable R_ARM_REL32 relocation against symbol `_ZTIi'
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

export "LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/alex.vongluck/Code/haiku/generated.arm/objects/linux/lib" ; ../build/scripts/rm_attrs objects/linux/x86_64/release/tools/rm_attrs -f "objects/haiku/arm/release/kits/libbe.so"
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/bin/arm-unknown-haiku-gcc -fno-strict-aliasing -fno-delete-null-pointer-checks -fno-builtin-fork -fno-builtin-vfork -march=armv7-a -mfloat-abi=hard -Xlinker --no-undefined -shared -Xlinker -soname="libbe.so" -nostdlib -Xlinker --no-undefined -o "objects/haiku/arm/release/kits/libbe.so"  "objects/haiku/arm/release/system/glue/arch/arm/crti.o" "/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtbeginS.o" "objects/haiku/arm/release/system/glue/init_term_dyn.o"  \
"objects/haiku/arm/release/kits/app/app_kit.o" "objects/haiku/arm/release/kits/interface/interface_kit.o" "objects/haiku/arm/release/kits/locale/locale_kit.o" "objects/haiku/arm/release/kits/storage/storage_kit.o" "objects/haiku/arm/release/kits/support/support_kit.o" "objects/haiku/arm/release/kits/storage/mime/libstorage_kit_mime.a" "objects/haiku/arm/release/libs/icon/libicon.a" "objects/haiku/arm/release/libs/agg/libagg.a" "build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libstdc++.so" "build_packages/icu-57.1-2-arm/develop/lib/libicudata.so" "build_packages/icu-57.1-2-arm/develop/lib/libicui18n.so" "build_packages/icu-57.1-2-arm/develop/lib/libicuio.so" "build_packages/icu-57.1-2-arm/develop/lib/libicule.so" "build_packages/icu-57.1-2-arm/develop/lib/libiculx.so" "build_packages/icu-57.1-2-arm/develop/lib/libicutu.so" "build_packages/icu-57.1-2-arm/develop/lib/libicuuc.so" "build_packages/zlib-1.2.11-1-arm/develop/lib/libz.so" "objects/haiku/arm/release/system/libroot/libroot.so" "build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libgcc_s.so.1" "build_packages/gcc_syslibs_devel-8.3.0_2019_05_24-1-arm/develop/lib/libgcc.a"  "/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtendS.o" "objects/haiku/arm/release/system/glue/arch/arm/crtn.o" \
-Wl,--version-script,../src/kits/libbe_versions

...failed Link objects/haiku/arm/release/kits/libbe.so ...

waddlesplash discovered that changing the linker to the following in th BuildConfig works around the issue:

HAIKU_LD_arm ?= /usr/bin/ld.lld ;
HAIKU_LINK_arm ?= /usr/bin/clang -fuse-ld=lld -target arm-unknown-freebsd -shared ;

[kallisti5]

I've confirmed that a swap of the linker solves this issue directly:

(-v added for verbosity between clang lld and gcc ld)

Stock:

$ /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/bin/arm-unknown-haiku-gcc -v -fno-strict-aliasing -fno-delete-null-pointer-checks -fno-builtin-fork -fno-builtin-vfork -march=armv7-a -mfloat-abi=hard -Xlinker --no-undefined -shared -Xlinker -soname="libbe.so" -nostdlib -Xlinker --no-undefined -o "objects/haiku/arm/release/kits/libbe.so"  "objects/haiku/arm/release/system/glue/arch/arm/crti.o" "/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtbeginS.o" "objects/haiku/arm/release/system/glue/init_term_dyn.o"  "objects/haiku/arm/release/kits/app/app_kit.o" "objects/haiku/arm/release/kits/interface/interface_kit.o" "objects/haiku/arm/release/kits/locale/locale_kit.o" "objects/haiku/arm/release/kits/storage/storage_kit.o" "objects/haiku/arm/release/kits/support/support_kit.o" "objects/haiku/arm/release/kits/storage/mime/libstorage_kit_mime.a" "objects/haiku/arm/release/libs/icon/libicon.a" "objects/haiku/arm/release/libs/agg/libagg.a" "build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libstdc++.so" "build_packages/icu-57.1-2-arm/develop/lib/libicudata.so" "build_packages/icu-57.1-2-arm/develop/lib/libicui18n.so" "build_packages/icu-57.1-2-arm/develop/lib/libicuio.so" "build_packages/icu-57.1-2-arm/develop/lib/libicule.so" "build_packages/icu-57.1-2-arm/develop/lib/libiculx.so" "build_packages/icu-57.1-2-arm/develop/lib/libicutu.so" "build_packages/icu-57.1-2-arm/develop/lib/libicuuc.so" "build_packages/zlib-1.2.11-1-arm/develop/lib/libz.so" "objects/haiku/arm/release/system/libroot/libroot.so" "build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libgcc_s.so.1" "build_packages/gcc_syslibs_devel-8.3.0_2019_05_24-1-arm/develop/lib/libgcc.a"  "/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtendS.o" "objects/haiku/arm/release/system/glue/arch/arm/crtn.o" -Wl,--version-script,../src/kits/libbe_versions
Using built-in specs.
COLLECT_GCC=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/bin/arm-unknown-haiku-gcc
COLLECT_LTO_WRAPPER=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/8.3.0/lto-wrapper
Target: arm-unknown-haiku
Configured with: /home/alex.vongluck/Code/buildtools/gcc/configure --prefix=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm --target=arm-unknown-haiku --disable-nls --disable-shared --with-system-zlib --enable-languages=c,c++ --enable-lto --enable-frame-pointer --enable-__cxa-atexit --enable-threads=posix --with-default-libstdcxx-abi=gcc4-compatible --with-sysroot=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/sysroot --disable-maintainer-mode --disable-multilib --with-float=hard --with-cpu=cortex-a8 --with-fpu=vfpv3 --disable-tls
Thread model: posix
gcc version 8.3.0 (GCC) 
COMPILER_PATH=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/8.3.0/:/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/8.3.0/:/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/:/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/:/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/:/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/
BELIBRARIES=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/:/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/lib/
COLLECT_GCC_OPTIONS='-v' '-fno-strict-aliasing' '-fno-delete-null-pointer-checks' '-fno-builtin-fork' '-fno-builtin-vfork'  '-mfloat-abi=hard' '-shared' '-nostdlib' '-o' 'objects/haiku/arm/release/kits/libbe.so' '-mfpu=vfpv3' '-marm' '-march=armv7-a+vfpv3'
 /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/8.3.0/collect2 -plugin /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/8.3.0/liblto_plugin.so -plugin-opt=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/libexec/gcc/arm-unknown-haiku/8.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccF1xVQF.res --sysroot=/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/sysroot -m armelf -shared -e 0 -X -o objects/haiku/arm/release/kits/libbe.so -L/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0 -L/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/lib --no-undefined -soname=libbe.so --no-undefined objects/haiku/arm/release/system/glue/arch/arm/crti.o /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtbeginS.o objects/haiku/arm/release/system/glue/init_term_dyn.o objects/haiku/arm/release/kits/app/app_kit.o objects/haiku/arm/release/kits/interface/interface_kit.o objects/haiku/arm/release/kits/locale/locale_kit.o objects/haiku/arm/release/kits/storage/storage_kit.o objects/haiku/arm/release/kits/support/support_kit.o objects/haiku/arm/release/kits/storage/mime/libstorage_kit_mime.a objects/haiku/arm/release/libs/icon/libicon.a objects/haiku/arm/release/libs/agg/libagg.a build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libstdc++.so build_packages/icu-57.1-2-arm/develop/lib/libicudata.so build_packages/icu-57.1-2-arm/develop/lib/libicui18n.so build_packages/icu-57.1-2-arm/develop/lib/libicuio.so build_packages/icu-57.1-2-arm/develop/lib/libicule.so build_packages/icu-57.1-2-arm/develop/lib/libiculx.so build_packages/icu-57.1-2-arm/develop/lib/libicutu.so build_packages/icu-57.1-2-arm/develop/lib/libicuuc.so build_packages/zlib-1.2.11-1-arm/develop/lib/libz.so objects/haiku/arm/release/system/libroot/libroot.so build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libgcc_s.so.1 build_packages/gcc_syslibs_devel-8.3.0_2019_05_24-1-arm/develop/lib/libgcc.a /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtendS.o objects/haiku/arm/release/system/glue/arch/arm/crtn.o --version-script ../src/kits/libbe_versions
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/ld: objects/haiku/arm/release/kits/interface/interface_kit.o: relocation R_ARM_REL32 against external or undefined symbol `_ZTIi' can not be used when making a shared object; recompile with -fPIC
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/ld: objects/haiku/arm/release/kits/interface/interface_kit.o(.ARM.extab+0x445c): unresolvable R_ARM_REL32 relocation against symbol `_ZTIi'
/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/../../../../arm-unknown-haiku/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
release/system/glue/arch/arm/crtn.o" -Wl,--version-script,../src/kits/libbe_versions

Updated:

/usr/bin/clang -v -fuse-ld=lld -target arm-unknown-freebsd -fno-strict-aliasing -fno-delete-null-pointer-checks -fno-builtin-fork -fno-builtin-vfork -march=armv7-a -mfloat-abi=hard -Xlinker --no-undefined -shared -Xlinker -soname="libbe.so" -nostdlib -Xlinker --no-undefined -o "objects/haiku/arm/release/kits/libbe.so"  "objects/haiku/arm/release/system/glue/arch/arm/crti.o" "/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtbeginS.o" "objects/haiku/arm/release/system/glue/init_term_dyn.o"  "objects/haiku/arm/release/kits/app/app_kit.o" "objects/haiku/arm/release/kits/interface/interface_kit.o" "objects/haiku/arm/release/kits/locale/locale_kit.o" "objects/haiku/arm/release/kits/storage/storage_kit.o" "objects/haiku/arm/release/kits/support/support_kit.o" "objects/haiku/arm/release/kits/storage/mime/libstorage_kit_mime.a" "objects/haiku/arm/release/libs/icon/libicon.a" "objects/haiku/arm/release/libs/agg/libagg.a" "build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libstdc++.so" "build_packages/icu-57.1-2-arm/develop/lib/libicudata.so" "build_packages/icu-57.1-2-arm/develop/lib/libicui18n.so" "build_packages/icu-57.1-2-arm/develop/lib/libicuio.so" "build_packages/icu-57.1-2-arm/develop/lib/libicule.so" "build_packages/icu-57.1-2-arm/develop/lib/libiculx.so" "build_packages/icu-57.1-2-arm/develop/lib/libicutu.so" "build_packages/icu-57.1-2-arm/develop/lib/libicuuc.so" "build_packages/zlib-1.2.11-1-arm/develop/lib/libz.so" "objects/haiku/arm/release/system/libroot/libroot.so" "build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libgcc_s.so.1" "build_packages/gcc_syslibs_devel-8.3.0_2019_05_24-1-arm/develop/lib/libgcc.a"  "/home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtendS.o" "objects/haiku/arm/release/system/glue/arch/arm/crtn.o" -Wl,--version-script,../src/kits/libbe_versions
clang version 8.0.0 (Fedora 8.0.0-1.fc30)
Target: arm-unknown-freebsd
Thread model: posix
InstalledDir: /usr/bin
 "/usr/bin/ld.lld" --eh-frame-hdr -Bshareable --enable-new-dtags -o objects/haiku/arm/release/kits/libbe.so -L/usr/lib --no-undefined -soname=libbe.so --no-undefined objects/haiku/arm/release/system/glue/arch/arm/crti.o /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtbeginS.o objects/haiku/arm/release/system/glue/init_term_dyn.o objects/haiku/arm/release/kits/app/app_kit.o objects/haiku/arm/release/kits/interface/interface_kit.o objects/haiku/arm/release/kits/locale/locale_kit.o objects/haiku/arm/release/kits/storage/storage_kit.o objects/haiku/arm/release/kits/support/support_kit.o objects/haiku/arm/release/kits/storage/mime/libstorage_kit_mime.a objects/haiku/arm/release/libs/icon/libicon.a objects/haiku/arm/release/libs/agg/libagg.a build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libstdc++.so build_packages/icu-57.1-2-arm/develop/lib/libicudata.so build_packages/icu-57.1-2-arm/develop/lib/libicui18n.so build_packages/icu-57.1-2-arm/develop/lib/libicuio.so build_packages/icu-57.1-2-arm/develop/lib/libicule.so build_packages/icu-57.1-2-arm/develop/lib/libiculx.so build_packages/icu-57.1-2-arm/develop/lib/libicutu.so build_packages/icu-57.1-2-arm/develop/lib/libicuuc.so build_packages/zlib-1.2.11-1-arm/develop/lib/libz.so objects/haiku/arm/release/system/libroot/libroot.so build_packages/gcc_syslibs-8.3.0_2019_05_24-1-arm/lib/libgcc_s.so.1 build_packages/gcc_syslibs_devel-8.3.0_2019_05_24-1-arm/develop/lib/libgcc.a /home/alex.vongluck/Code/haiku/generated.arm/cross-tools-arm/lib/gcc/arm-unknown-haiku/8.3.0/crtendS.o objects/haiku/arm/release/system/glue/arch/arm/crtn.o --version-script ../src/kits/libbe_versions

[tqh]

It looks like there are XXX's and TODO's in our linker scripts, has anyone any clue how correct those are? ​https://github.com/haiku/haiku/tree/master/src/system/ldscripts/arm

[adamfowleruk]

I just got this compiling successfully on the latest Manjaro Linux.

I changed this:

HAIKU_LD_arm ?= /home/adam/Documents/git/haiku/generated.arm/cross-tools-arm/bin/arm-unknown-haiku-ld ;

To this:-

HAIKU_LD_arm ?= /usr/bin/ld.lld ;
HAIKU_LINK_arm ?= /usr/bin/clang -fuse-ld=lld -target arm-unknown-linux -shared ;

Note the Linux instead of freebsd.

I also installed clang, lld, uboot-utils and dtc packages for Manjaro using pacman.

It appears to have compiled and built an image. Now to test it :/

[simonsouth]

The underlying issue is that gcc is ​configured at build time to request ld use its standard armelf emulation rather than Haiku's own variant, ​armelf_haiku.

Among other things this means ​the TARGET2_TYPE linker option isn't being set, causing ld to transparently map R_ARM_TARGET2 relocations to R_ARM_REL32 (the built-in default, apparently). This effectively specifies static relocation, which presumably is why ld suggests recompiling with -fPIC: If it finds a symbol can't be relocated it must assume the code was built without relocation in mind.

Since Haiku always provides dynamic relocation at runtime R_ARM_TARGET2 should instead map to R_ARM_GOT_PREL, one of the things the armelf_haiku emulation specifies.

Fixing gcc's build configuration and rebuilding the cross-tools allows the ARM build of Haiku to complete, and seems to reduce the number of compiler warnings, too.

I'll upload a patch shortly but the only change is to modify ​gcc's LINKSPEC to specify "-m armelf_haiku" instead of "-m armelf".

[simonsouth]

I've uploaded a patch to Gerrit for review: ​https://review.haiku-os.org/c/buildtools/+/1702

[waddlesplash]

Patch merged in btrev43151. Thanks!

[nielx]

Assign tickets with status=closed and resolution=fixed within the R1/beta2 development window to the R1/beta2 Milestone