Issues with __cxa_atexit

[kaliber]

First of all thanks for the _cxa_atexit implementation. Unfortunately many clang apps crash at exit. Here is an example:

[Switching to team /boot/system/servers/print_server (103) thread print_server (103)]
0x002f4ea1 in __cxa_atexit (hook=0x680260 <std::ios_base::Init::~Init()>, 
    data=0x19d0268, dsoHandle=0x19d01b0)
    at src/system/libroot/posix/stdlib/exit.cpp:220
220     src/system/libroot/posix/stdlib/exit.cpp: No such file or directory.
        in src/system/libroot/posix/stdlib/exit.cpp
(gdb) thread apply all bt

Thread 1 (team /boot/system/servers/print_server (103) thread print_server (103)):
#0  0x002f4ea1 in __cxa_atexit (hook=0x680260 <std::ios_base::Init::~Init()>, 
    data=0x19d0268, dsoHandle=0x19d01b0)
    at src/system/libroot/posix/stdlib/exit.cpp:220
#1  0x019c77fe in global constructors keyed to a ()
   from /boot/system/add-ons/Print/transport/IPP
#2  0x019ca557 in __do_global_ctors_aux ()
   from /boot/system/add-ons/Print/transport/IPP
#3  0x019c0609 in _init () from /boot/system/add-ons/Print/transport/IPP
#4  0x00100878 in init_dependencies () from /boot/system/runtime_loader
#5  0x00100d78 in load_library () from /boot/system/runtime_loader
#6  0x00103b17 in export_load_add_on () from /boot/system/runtime_loader
#7  0x00260c64 in load_add_on (
    name=0x18022b30 "/boot/system/add-ons/Print/transport/IPP")
    at src/system/libroot/os/image.cpp:73
#8  0x002182d3 in Transport::Transport ()
#9  0x002181e0 in Transport::Scan ()
#10 0x00210f57 in PrintServerApp::PrintServerApp ()
#11 0x00210e03 in main ()

[bonefish]

Probably fixed in hrev39324. Please give it a try (you only need to update libroot.so).

[kaliber]

The first issue has been fixed. Thanks! There is a second issue with print_server during system shutdown:

0x01af4a60 in ?? ()
(gdb) bt
#0 0x01af4a60 in ?? ()
#1 0x002f5032 in __cxa_finalize (dsoHandle=0x0)
   at src/system/libroot/posix/stdlib/exit.cpp:256
#2 0x002f5395 in exit (status=0)
   at src/system/libroot/posix/stdlib/exit.cpp:310
#3 0x00210816 in _start ()
#4 0x001058a2 in runtime_loader () from /boot/system/runtime_loader

[bonefish]

My guess is that __cxa_finalize() is not called when add-ons are unloaded (hence the exit hooks for already unloaded add-ons are invoked when the program exits). AFAIK it is the compiler's responsibility to put such a call in the termination routine of a shared object. Please add one of the print server add-ons to the ticket (one that calls __cxa_atexit() -- check with nm), so I can examine it.

[kaliber]

PostScript print add-on (clang)

[bonefish]

That verifies my suspicion. __cxa_finalize() isn't called by __do_global_dtors_aux(). This is a compiler/tool chain problem. When using gcc/binutils the respective code seems to stem from crtbegin.o/crtend.o. If you're still mixing clang and parts of gcc/binutils, then you might incorrectly use those files, too.

[kaliber]

In my opinion tool chain call looks fine:

clang --verbose -fno-strict-aliasing -fno-tree-vrp -Xlinker --no-undefined -nostart -Xlinker -soname="PS Compatible" -nostdlib -Xlinker --no-undefined -o "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS Compatible"  "generated/objects/haiku/x86/release/system/glue/arch/x86/crti.o" "/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtbegin.o" "generated/objects/haiku/x86/release/system/glue/init_term_dyn.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSEntry.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSCap.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSData.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/SelectPPDDlg.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PPDParser.o" "generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/FilterIO.o" "generated/objects/haiku/x86/release/kits/libbe.so" "generated/objects/haiku/x86/release/libs/print/libprint/libprint.a" "/boot/system/lib/libstdc++.so" "generated/objects/haiku/x86/release/system/libroot/libroot.so"  "/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtend.o" "generated/objects/haiku/x86/release/system/glue/arch/x86/crtn.o"
clang version 2.9 (trunk 118327)
Target: i386-pc-haiku
Thread model: posix
 "/boot/develop/tools/gnupro/bin/gcc" -v -fno-strict-aliasing -fno-tree-vrp -nostart -nostdlib -m32 -o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS Compatible -Xlinker --no-undefined -Xlinker -soname=PS Compatible -Xlinker --no-undefined generated/objects/haiku/x86/release/system/glue/arch/x86/crti.o /boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtbegin.o generated/objects/haiku/x86/release/system/glue/init_term_dyn.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSEntry.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSCap.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSData.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/SelectPPDDlg.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PPDParser.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/FilterIO.o generated/objects/haiku/x86/release/kits/libbe.so generated/objects/haiku/x86/release/libs/print/libprint/libprint.a /boot/system/lib/libstdc++.so generated/objects/haiku/x86/release/system/libroot/libroot.so /boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtend.o generated/objects/haiku/x86/release/system/glue/arch/x86/crtn.o
Using built-in specs.
Target: i586-pc-haiku
Configured with: ../gcc/configure --target=i586-pc-haiku --disable-nls --enable-languages=c,c++ --disable-multilib --prefix=/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705
Thread model: single
gcc version 4.4.4 (GCC) 
COMPILER_PATH=/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/libexec/gcc/i586-pc-haiku/4.4.4/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/libexec/gcc/i586-pc-haiku/4.4.4/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/libexec/gcc/i586-pc-haiku/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/../../../../i586-pc-haiku/bin/
BELIBRARIES=/boot/develop/abi/current/library-paths/common/:/boot/develop/lib/x86/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/../../../../i586-pc-haiku/lib/:/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/../../../
COLLECT_GCC_OPTIONS='-v' '-fno-strict-aliasing' '-fno-tree-vrp' '-nostart' '-nostdlib' '-m32' '-o' 'generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS Compatible' '-mtune=pentium'
 /boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/libexec/gcc/i586-pc-haiku/4.4.4/collect2 -m elf_i386_haiku -shared -Bsymbolic -e 0 -o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS Compatible -L/boot/develop/abi/current/library-paths/common -L/boot/develop/lib/x86 -L/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4 -L/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/../../../../i586-pc-haiku/lib -L/boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/../../.. --no-undefined -soname=PS Compatible --no-undefined generated/objects/haiku/x86/release/system/glue/arch/x86/crti.o /boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtbegin.o generated/objects/haiku/x86/release/system/glue/init_term_dyn.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSEntry.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PS.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSCap.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PSData.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/SelectPPDDlg.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/PPDParser.o generated/objects/haiku/x86/release/add-ons/print/drivers/postscript/FilterIO.o generated/objects/haiku/x86/release/kits/libbe.so generated/objects/haiku/x86/release/libs/print/libprint/libprint.a /boot/system/lib/libstdc++.so generated/objects/haiku/x86/release/system/libroot/libroot.so /boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtend.o generated/objects/haiku/x86/release/system/glue/arch/x86/crtn.o

Shouldn't dlclose() call/register "atexit"? LSB specification ​http://refspecs.freestandards.org/LSB_3.2.0/LSB-Core-generic/LSB-Core-generic/baselib---cxa_finalize.html says: "The implementation shall arrange for _cxa_finalize() to be called during early shared library unload (e.g. dlclose()) with a handle to the shared library".

[bonefish]

Replying to kaliber:

In my opinion tool chain call looks fine:

Nope, it doesn't. As you can see the object is linked with /boot/develop/abi/x86/gcc4/tools/gcc-4.4.4-haiku-100705/lib/gcc/i586-pc-haiku/4.4.4/crtbegin.o. That file belongs to the gcc distribution. It contains the function __do_global_dtors_aux() which would call __cxa_finalize(), if gcc would register any destructors via __cxa_atexit(). Since our gcc currently doesn't do the latter it also doesn't do the former. If clang uses __cxa_atexit(), it has to make sure that __cxa_finalize() is called -- it therefore cannot use gcc's crtbegin.o. At least not until we enable that feature in gcc as well.

Shouldn't dlclose() call/register "atexit"? LSB specification ​http://refspecs.freestandards.org/LSB_3.2.0/LSB-Core-generic/LSB-Core-generic/baselib---cxa_finalize.html says: "The implementation shall arrange for _cxa_finalize() to be called during early shared library unload (e.g. dlclose()) with a handle to the shared library".

The term "implementation" in the Itanium C++ ABI specification mainly refers to the compiler/tool chain. In this case gcc definitely seems to feel responsible as it calls __cxa_finalize() in its ​crtstuff.c. We just don't build the crtbegin.o version which would do that.

[kaliber]

Thanks for explanation. I've fixed gcc and haiku to use 'crt' code for shared libraries (crtbeginS.o, crtendS.o) and rebuilt haiku. It works now correctly. Should it be enabled for gcc4 by default?

[bonefish]

I see two issues with the patches:

• I doubt the gcc patch is sufficient. By using the shared library glue code __cxa_finalize() will be called, but since gcc hasn't ever output code using __cxa_atexit() I suspect it won't do that now either. So it probably has to be enabled explicitly.
• Selecting the glue code depending on the shared option has the problem that it won't work for executable add-ons -- __cxa_finalize() will not be called for them.

PS: It's not so good an idea to attach patches to a closed ticket.

[kaliber]

Replying to bonefish:

I see two issues with the patches:

• I doubt the gcc patch is sufficient. By using the shared library glue code __cxa_finalize() will be called, but since gcc hasn't ever output code using __cxa_atexit() I suspect it won't do that now either. So it probably has to be enabled explicitly.

I think that --enable-__cxa_atexit is sufficient. I've enabled it by default for Haiku in gcc/config.gcc

• Selecting the glue code depending on the shared option has the problem that it won't work for executable add-ons -- __cxa_finalize() will not be called for them.

Are there executable and non-executable add-ons? For example filetype add-on for tracker is build with -nostart (-shared), so it will work. If there are executable add-ons, we can introduce -addon flag for gcc.

[kaliber]

I updated cxa-haiku.patch, because it had incorrect value for HAIKU_EXECUTABLE_END_GLUE_CODE.

[bonefish]

Replying to kaliber:

Replying to bonefish:

I see two issues with the patches:

• I doubt the gcc patch is sufficient. By using the shared library glue code __cxa_finalize() will be called, but since gcc hasn't ever output code using __cxa_atexit() I suspect it won't do that now either. So it probably has to be enabled explicitly.

I think that --enable-__cxa_atexit is sufficient. I've enabled it by default for Haiku in gcc/config.gcc

Uh, sorry, I don't know how I missed that.

• Selecting the glue code depending on the shared option has the problem that it won't work for executable add-ons -- __cxa_finalize() will not be called for them.

Are there executable and non-executable add-ons? For example filetype add-on for tracker is build with -nostart (-shared), so it will work.

Replicants come to mind. They are usually implemented directly in executables, but need to be loaded as an add-on by the host application. There are also regular add-ons that are executable. Most of those don't provide any actual functionality this way, though, but only show an about box or some alert.

If there are executable add-ons, we can introduce -addon flag for gcc.

I haven't read up on that, but -pie might already be supposed to have the desired semantics. It's just not properly implemented in the gcc specs yet.

[jessicah]

I believe olta did some work related to this. So is this still a problem, or can we now close it?

[waddlesplash]

Yes, this was fixed. If it reappears we can create a new ticket.