Opened 13 years ago
Closed 13 years ago
#8031 closed bug (fixed)
new gcc indirect linking policy changes cause build failures / missing symbols.
Reported by: | kallisti5 | Owned by: | bonefish |
---|---|---|---|
Priority: | high | Milestone: | R1/beta1 |
Component: | Build System | Version: | R1/Development |
Keywords: | Cc: | ||
Blocked By: | Blocking: | ||
Platform: | All |
Description
Haiku gcc4 build fails on Ubuntu 11.10.
gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)
attached is full build output.
generated/objects/linux/x86/release/tools/unzip/libunzip.a(beos.o): In function `setBeOSexfield': beos.c:(.text+0x43e): undefined reference to `_kern_open' beos.c:(.text+0x494): undefined reference to `__swap_int32' beos.c:(.text+0x4a5): undefined reference to `__swap_int64' beos.c:(.text+0x599): undefined reference to `fs_write_attr' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_open_dir_entry_ref' generated/objects/linux/lib/libbe_build.so: undefined reference to `_haiku_build_strerror' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_unlink' generated/objects/linux/lib/libbe_build.so: undefined reference to `strlcpy' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_read_stat' generated/objects/linux/lib/libbe_build.so: undefined reference to `atomic_get' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_open_attr_dir' generated/objects/linux/lib/libbe_build.so: undefined reference to `__gUmask' generated/objects/linux/lib/libbe_build.so: undefined reference to `create_sem' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::~KMessage()' generated/objects/linux/lib/libbe_build.so: undefined reference to `_haiku_build_errno' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::GetNextField(BPrivate::KMessageField*) const' generated/objects/linux/lib/libbe_build.so: undefined reference to `fs_stat_attr' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_remove_attr' generated/objects/linux/lib/libbe_build.so: undefined reference to `find_thread' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_create_dir' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessageField::TypeCode() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_read' generated/objects/linux/lib/libbe_build.so: undefined reference to `delete_sem' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_write' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::ReplyToken() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::ReplyPort() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_unlock_node' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_entry_ref_to_path' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::KMessage()' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::TargetToken() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_rename_attr' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessageField::ElementAt(int, int*) const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_rewind_dir' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_read_link' generated/objects/linux/lib/libbe_build.so: undefined reference to `__swap_int16' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessageField::Name() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_open_dir' generated/objects/linux/lib/libbe_build.so: undefined reference to `atomic_add' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_open_parent_dir' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_read_dir' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessageField::KMessageField()' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_rename' generated/objects/linux/lib/libbe_build.so: undefined reference to `release_sem' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::SetTo(void const*, int, unsigned int)' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::What() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_close' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessage::kMessageHeaderMagic' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_lock_node' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_seek' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_open_entry_ref' generated/objects/linux/lib/libbe_build.so: undefined reference to `debugger' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessageField::HasFixedElementSize() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `fs_read_attr' generated/objects/linux/lib/libbe_build.so: undefined reference to `acquire_sem_etc' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_fsync' generated/objects/linux/lib/libbe_build.so: undefined reference to `BPrivate::KMessageField::CountElements() const' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_create_symlink' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_write_stat' generated/objects/linux/lib/libbe_build.so: undefined reference to `_kern_dup'
Attachments (7)
Change History (33)
by , 13 years ago
Attachment: | failed-build-r42873.txt added |
---|
by , 13 years ago
Attachment: | UserBuildConfig added |
---|
comment:1 by , 13 years ago
Keywords: | 11.10 libunzip.a setBeOSexfield undefined reference removed |
---|
On a quick read through the gcc 4.6 release changes I didn't see anything related. What's the binutils version? Please also list the symbols of libroot_build (readelf --syms ...
).
comment:2 by , 13 years ago
kallisti5@houvonglucka23:~$ ar -V GNU ar (GNU Binutils for Ubuntu) 2.21.53.20110810 Copyright 2011 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) any later version. This program has absolutely no warranty.
readelf output attached
by , 13 years ago
Attachment: | libbe_symbols.txt added |
---|
by , 13 years ago
Attachment: | libroot_symbols.txt added |
---|
comment:3 by , 13 years ago
The link line lists libroot_build.so
and the library defines and exports the symbols in question, so I don't see why the linker would complain. I didn't spot anything related in the binutils changelog either.
You could experiment a bit with the link line -- e.g. with the library order -- and see if that makes a difference. Also run with -v
so we see the tools invoked.
follow-up: 5 comment:4 by , 13 years ago
Hi, I built gcc4/gcc2 hybrid on Ubuntu 11.10 X86-32. I encountered same unzip problem, and adding $(HOST_LIBROOT) in unzip/Jamfile solved problem (other part except unzip can build well).
diff --git a/src/tools/unzip/Jamfile b/src/tools/unzip/Jamfile index 6ca3b0a..79b9db4 100644 --- a/src/tools/unzip/Jamfile +++ b/src/tools/unzip/Jamfile @@ -31,6 +31,6 @@ BuildPlatformMain unzip : unzip.c unreduce.c unshrink.c - : libunzip.a $(HOST_LIBBE) $(HOST_LIBSUPC++) + : libunzip.a $(HOST_LIBBE) $(HOST_LIBROOT) $(HOST_LIBSUPC++) ;
Another problem: gcc2 build tools could not build with gcc 4.6.1 in Ubuntu 11.10, so I need to install gcc-4.5 package and confirured with CC=gcc-4.5 .
comment:5 by , 13 years ago
Replying to mt:
Hi, I built gcc4/gcc2 hybrid on Ubuntu 11.10 X86-32. I encountered same unzip problem, and adding $(HOST_LIBROOT) in unzip/Jamfile solved problem (other part except unzip can build well).
Can you show me the resulting link line (near the end of the jam -na '<build>unzip'
output)? As we see in kallisti5's output, libroot_build.so
is already linked against. Adding $(HOST_LIBROOT)
would just add it a second time. So I wonder, if the link order is somehow relevant, although it shouldn't be for shared libraries. This smells a bit like a linker (ld or gold, whichever is used) bug.
Another problem: gcc2 build tools could not build with gcc 4.6.1 in Ubuntu 11.10, so I need to install gcc-4.5 package and confirured with CC=gcc-4.5 .
The gcc 2 sources are rather old and we've repeatedly seen issues with newer compiler versions. They usually could be fixed without too much work. Patches welcome, of course. :-)
by , 13 years ago
Attachment: | link-verbose.txt added |
---|
comment:6 by , 13 years ago
confirmed what bonefish said. reversing the two shared libraries in the linking fixed the build...
follow-up: 10 comment:7 by , 13 years ago
#gcc:
09:40 < Zordan> if you are using e.g. newer gold linker (and perhaps newer regular ld as well?) then indirect shared library dependencies are no longer followed (e.g. shared libraries have to list their own direct dependencies)
comment:8 by , 13 years ago
#gcc:
09:44 < Zordan> kallisti5: maybe similar in change to what fedora made the default as well: https://fedoraproject.org/wiki/UnderstandingDSOLinkChange 09:48 < Zordan> apparently also in ubuntu since natty: https://wiki.ubuntu.com/NattyNarwhal/ToolchainTransition
comment:9 by , 13 years ago
Summary: | Build fails on Ubuntu 11.10 x86_64 → new gcc indirect linking policy changes cause build failures / missing symbols. |
---|
follow-up: 12 comment:10 by , 13 years ago
Replying to kallisti5:
#gcc:
09:40 < Zordan> if you are using e.g. newer gold linker (and perhaps newer regular ld as well?) then indirect shared library dependencies are no longer followed (e.g. shared libraries have to list their own direct dependencies)
The first part of this statement seems to refer to what is explained in the "UnderstandingDSOLinkChange" wiki page. It basically says that, if your app uses a library function, it has to be linked directly to that library and cannot rely on another library pulling in the first library. This makes sense. (BTW, it is something that was mandatory on BeOS as well, as the runtime loader did support resolving symbols only in direct dependencies.)
The part in the last parentheses is a bit ambiguous. It might either mean when a library is linked, it has to be linked against all its dependencies or that when linking against a library all its dependencies have to be listed as well. The latter I would find rather ridiculous.
Anyway, we do list libroot_build.so, both when linking libbe_build.so as well as when linking the app, so either way any condition the statement might refer to is fulfilled.
What apparently seems to be the problem is that the linker (the verbose output doesn't show which one is actually invoked by collect2 -- the error message says "ld returned 1 exit status", but that might not refer to the actual name of the linker) even requires the libraries to be listed in dependency order. That I would consider a bug or at least a seriously missing feature.
by , 13 years ago
Attachment: | collect2-verbose.txt added |
---|
by , 13 years ago
Attachment: | collect2-verbose-success.txt added |
---|
comment:11 by , 13 years ago
swapping the shared library orders in collect2 also resolves the issue.
Verbose failure, original sorting (collect2-verbose.txt) Verbose success, modified shared library sorting (collect2-verbose-success.txt)
comment:12 by , 13 years ago
Replying to bonefish:
What apparently seems to be the problem is that the linker (...) even requires the libraries to be listed in dependency order. That I would consider a bug or at least a seriously missing feature.
It's by design. The second link has it in there:
Also in Natty, ld runs with the --as-needed option enabled by default. ... The --as-needed option also makes the linker sensitive to the ordering of libraries on the command-line. You may need to move some libraries later in the command-line, so they come after other libraries or files that require symbols from them.
Basically --as-needed links in only actually required libraries. Since that lookup obviously only goes in one direction it's pretty much the same as for static libraries now.
follow-up: 14 comment:13 by , 13 years ago
mmlr: confirmed. removing --as-needed from collect made it link properly. Adding '--no-as-needed' also made collect link properly.
So maybe setting a linker flag of '--no-as-needed' in the build... or are we just covering a bug in our build process?
comment:14 by , 13 years ago
Replying to kallisti5:
mmlr: confirmed. removing --as-needed from collect made it link properly. Adding '--no-as-needed' also made collect link properly.
Did this require changes to the jam build files? If so, for the jam newbies out there (ie, me), could you post a diff? Thanks.
comment:15 by , 13 years ago
If mt's change from #comment:4 fixes the build, why don't we just do that? Assuming of course it doesn't cause issues on Haiku or other build platforms. I'll probably be setting up Ubuntu soon on a new computer and would like to be able to build Haiku on it.
follow-up: 17 comment:16 by , 13 years ago
leavengood: yeah, we can do that... however it's a hack. $(HOST_LIBROOT) is already linked.. but in the wrong order...
then again.. having --no-as-needed --as-needed --no-as-needed --as-needed --no-as-needed in the final compile statement is kinda a hack as well :)
I'll add it.
comment:17 by , 13 years ago
Replying to kallisti5:
leavengood: yeah, we can do that... however it's a hack. $(HOST_LIBROOT) is already linked.. but in the wrong order...
then again.. having --no-as-needed --as-needed --no-as-needed --as-needed --no-as-needed in the final compile statement is kinda a hack as well :)
Not really. It would just counter a flag that really shouldn't be there in the first place. I agree with the general intend of avoiding building in library dependencies that aren't necessary, but --as-needed
implements it in a way that makes using the linker unnecessarily inconvenient. I find it rather stupid to list libraries multiple times just to appease such a linker.
follow-up: 19 comment:18 by , 13 years ago
bonefish: libroot was *just* added in hrev43186 to unzip build, confirmed working on gcc4 Ubuntu 11.10
alternate solutions? I couldn't find any flags that would reliably work. really we need to remove the new DSO flags... but thats hard to do as they are compiled in.
Maybe a gcc spec file?
comment:19 by , 13 years ago
Replying to kallisti5:
alternate solutions? I couldn't find any flags that would reliably work.
Huh? In comment:13 you write that adding --no-as-needed
works. I don't see why adding -Xlinker --no-as-needed
to HOST_LINKFLAGS
wouldn't work. If it doesn't, please post the error and command line.
really we need to remove the new DSO flags... but thats hard to do as they are compiled in.
There's a very simple solution: Build ld respectively gcc without this flag enabled. I haven't found in either release notes that the flag has been enabled by default, so I suspect that's just a "brilliant" move of the Ubuntu makers. Complaining to them might help.
comment:20 by , 13 years ago
bonefish: yeah. I think I was mistaken on the --no-as-needed change due to the unzip lib already being built.
Let me do some testing minus hrev43186 real quick and post here.
comment:21 by , 13 years ago
verified that I was incorrect before...
pre hrev43186:
- stock: build failure
- add --no-as-needed: build failure
- add --as-needed: build failure
- add --no-add-needed: build failure
- add --add-needed: build failure
the stock collect command where the issue is:
/usr/lib/gcc/x86_64-linux-gnu/4.6.1/collect2 --build-id --no-add-needed --as-needed --eh-frame-hdr -m elf_x86_64 --hash-style=gnu -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o ../../../generated/objects/linux/x86/release/tools/unzip/unzip /usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.6.1/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1 -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../.. ../../../generated/objects/linux/x86/release/tools/unzip/unzip.o ../../../generated/objects/linux/x86/release/tools/unzip/unreduce.o ../../../generated/objects/linux/x86/release/tools/unzip/unshrink.o ../../../generated/objects/linux/lib/libroot_build.so ../../../generated/objects/linux/x86/release/tools/unzip/libunzip.a ../../../generated/objects/linux/lib/libbe_build.so -lsupc++ -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/4.6.1/crtend.o /usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../x86_64-linux-gnu/crtn.o
notice the multiple -needed flags. Ubuntu and Fedora seem to do this now with Debian considering it.
- Removing the first --as-needed causes the collect to succeed... adding more flags doesn't seem to disable it.
comment:22 by , 13 years ago
I see people dealing with this by writing out a modified gcc spec file... seems like a pain to automate though:
http://permalink.gmane.org/gmane.comp.clustering.open-mpi.user/14707
comment:23 by , 13 years ago
Can you please show me the link line produced by our build system (I mean the cc ...
; the resulting collect2 invocation might be interesting, too) when is -Xlinker --no-as-needed
is added to HOST_LINKFLAGS
.
comment:24 by , 13 years ago
Could we just keep the somewhat "hacky" fix from hrev43186, even if it isn't totally correct, since this only seems to affect one component? I'm not sure the time investment is worth it now due to one build platform's bad decision.
comment:25 by , 13 years ago
Well I guess I spoke up too late, given hrev43191. Still it might be good to look further into why this was set up on Ubunutu versus just turning off this option globally in our build system. Though it does seem like a stupidly implemented GCC option.
comment:26 by , 13 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
sigh.
seems you were right bonefish, gcc is smart enough and pick up the removal through Xlinker and remove the excess arguments when calling collect.
fixed *properly* in hrev43191. I added the HOST_LINKFLAGS addition below HOST_GCC_VERSION in the BuildSetup file in case we need to check the host gcc version at some point. (i have no idea when --no-as-needed was introduced)
resolved.
full build output