Support for x86-64 architecture

[ekdahl]

Port Haiku to the x86-64 architecture. Tasks:

• Add cross-compilation support for Haiku/x86-64 to gcc 4.
• Add ELF64 support to boot loader and kernel.
• Extend the x86 boot loader to be able to start an x86-64 kernel.
• Port the kernel itself, i.e. write the x86-64 specific kernel code (CPU, VM, etc.).
• Port fundamental kernel modules to 64 bit/x86-64.
• Port libroot.so and runtime loader to x86-64.
• Port other userland libraries and applications to x86-64.

[wkornewald]

I've at least renamed it here in Trac. :)

[nmentley]

I've attached a zip file including a patch a few other files that'll allow gcc4 to compile with a haiku-pc-x86_64 target.

[mmadia]

For patches & in general, it's preferred to submit them as plaintext. SubmittingPatches details how to do so.

[nmentley]

Sorry for uploading the patch in the wrong format. I remade the patch files against the newest svn revision and re-uploaded them as plan text.

[bonefish]

Thanks for the patches! A few remarks regarding the build tools changes:

• binutils/ld/configure.tgt, binutils/ld/Makefile.in:
  • Most OSs seem to use the elf_x86_64 target. Does Haiku need its own? At least binutils/ld/emulparams/elf_x86_64_haiku.sh is functionally identical to binutils/ld/emulparams/elf_x86_64.sh.
  • We want targ_extra_emuls=..., so we get x86 support in the x86_64 gcc.
• binutils/gas/configure.tgt:
  • Please don't make coding style changes (even if they are fitting). Until we upstream our patches, we want to keep them as small as possible to avoid merge conflicts.
  • The "i386-*-haiku*)" change is superfluous. In both cases the same is done as before.
• binutils/bfd/config.bfd:
  • Pretty much all other targets set the targ_selvecs variable. I haven't checked what it means, but I could imagine, we want it too. Maybe it's similar to the targ_extra_emuls in ld.

• gcc/gcc/config.gcc:
  • Including both i386/haiku.h and i386/haiku64.h doesn't seem right. There's quite a bit of duplication.
• gcc/gcc/config/i386/haiku64.h:
  • DEFAULT_PCC_STRUCT_RETURN: There's an interesting comment in linux64.h. We should find out whether this (already) applies to the Haiku x86_64 target as well.
  • TARGET_OS_CPP_BUILTINS: I don't think we need to define _X86_64_. TARGET_CPU_CPP_BUILTINS already defines a whole bunch of macros identifying x86_64 (__x86_64[__], __amd64[__]). I would also drop __INTEL__. headers/config/HaikuConfig.h, headers/build/config_build/HaikuConfig.h (and maybe other headers) and build/jam/BuildSetup need to be adjusted then, though.

Regarding the Haiku changes:

• configure:
  • Why the --use-64bit option? HAIKU_HOST_USE_32BIT is only relevant for the host platform. You shouldn't need to change it for the x86_64 target.
• The files in headers/posix/arch/x86_64 seem to be copies of the x86 files. I'm afraid there's a bit more that has to be done -- x86_64 has a bunch of new registers for instance.

[nmentley]

Thanks, for the comments!

Sorry about the delay on working on this, I've been finish up finals and moving off campus. Also, I'm not exactly sure where I should be asking development questions like this... so if this is the wrong spot let me know and I'll try to avoid making this mistake again.

The comments were extremely helpful, but I do have a question or two. I'm curious what the most effective way would be to test the toolchain.

I also happen to be a bit lost on the (arch)/signal.h header. I've been looking at the i386 signal.h as an example, but I am a bit confused. In vregs the ebx register isn't even referenced and I'm not exactly sure what purpose _reserved_1 and _reserved_3 serve.

looking at the arm port vregs just lists the general purpose registers... so that's what I currently have for x86_64, but I feel like I should have a better understanding of the i386 file to list the sse and the mmx registers.

Thanks

[bonefish]

Replying to nmentley:

Sorry about the delay on working on this, I've been finish up finals and moving off campus.

No worries.

Also, I'm not exactly sure where I should be asking development questions like this... so if this is the wrong spot let me know and I'll try to avoid making this mistake again.

The ticket is OK, if it is about the attached patch[es] -- though it gets unhandy when the number of comment grows. The best place for discussion is one of the mailing lists. Being a GSoC project the haiku-gsoc list is fine, or generally haiku-development if you prefer that one. It's also fine to send the patches there, though it doesn't harm to attach them to a ticket as well.

The comments were extremely helpful, but I do have a question or two. I'm curious what the most effective way would be to test the toolchain.

I suppose the best way is to compile (parts of) Haiku. It doesn't necessarily have to be perfect before you continue, anyway. If you discover problems later you can just go back an fix them.

I also happen to be a bit lost on the (arch)/signal.h header. I've been looking at the i386 signal.h as an example, but I am a bit confused. In vregs the ebx register isn't even referenced and I'm not exactly sure what purpose _reserved_1 and _reserved_3 serve.

The structure looks the same in BeOS. Supposedly those fields where once used and are not needed anymore or just reserved space for future extensions (changing structure size would result in binary compatibility issues). For the purpose of vregs have a look at the documentation in headers/posix/signal.h. Why one would want to have the feature only for those registers, I don't know. Haiku uses the reserved fields to also store the other general registers (ebx, esi, edi), BTW.

looking at the arm port vregs just lists the general purpose registers... so that's what I currently have for x86_64, but I feel like I should have a better understanding of the i386 file to list the sse and the mmx registers.

You don't need to focus on that yet. My remark was more of a general nature (mainly because your patch doesn't suggest that the file still needs work). The signal stuff can be revised when the kernel gets to the points where the initial shell is started. A TODO will do just fine for the time being.

I would recommend to rework the patch considering my earlier comments and then set out to get the boot loader (haiku_loader) and the kernel building. The next step could be to build an image containing these two and start working on getting the boot loader to actually load and start the kernel.

[nmentley]

added a few header files in haiku/headers/posix/arch that'll allow for an x86_64 haiku target to be compiled. Also edited ./configure to support x86_64 arch when running build-cross-tools-gcc4

[nmentley]

Thanks, I've noticed that the fenv.h files in /headers/ponix/arch/ are from freebsd. The x86_64 one i've included is also from freebsd. I've left the original header of the file intact... Is there anything else I should do when including code from other projects?

[bonefish]

Replying to nmentley:

Thanks, I've noticed that the fenv.h files in /headers/ponix/arch/ are from freebsd. The x86_64 one i've included is also from freebsd. I've left the original header of the file intact... Is there anything else I should do when including code from other projects?

When larger components are included we use a vendor branch, but individual files we just add. The exact version/revision of the file should be given in commit message, though, so future updates will be simpler (in case we make changes).

Some more comments on the patches:

• binutils/gas/configure.tgt:
  • Only changes whitespace. Please remove.
• binutils/bfd/config.bfd:
  • I don't think we need the i386coff_vec in targ_selvecs. Haiku doesn't support COFF.
• gcc/gcc/config/i386/haiku64.h:
  • typo in TODO: reoved -> removed
  • I don't understand the comment "If ELF is the default format, we should not use /lib/elf". You probably copied it from somewhere, but Haiku doesn't have a /lib/elf, actually not even a /lib.
• build/jam/BuildSetup:
  • bios_x86-64 -> bios_x86_64. That's the name of the boot platform directory in src/system/boot/platform. By convention we use underscores only.
  • The TODO in the last x86_64 case is superfluous. We don't need -march, since we want to support all x86_64 platforms. You could add HAIKU_DEFINES += B_USE_BUILTIN_ATOMIC_FUNCTIONS, though. The atomic built-ins should be available for gcc4. They just aren't enabled for the platforms != x86 yet, because they have not been tested there. In fact I would even try and implement the atomic_*() functions for x86_64 using the built-ins. Saves you some meddling with assembly code.
• headers/posix/arch/x86_64/arch_setjmp.h:
  • This one looks unfinished (12 ints probably won't suffice). That's OK for starters, but please add a TODO at least. Note that setjmp()/longjmp() are used in the kernel debugger, so you'll need to implement them relatively early.

Other than that the patches look OK. Please address the comments, then I'll commit them.

[nmentley]

edited configure files in the buildtools that'll allow for an x86_64 haiku target to be compiled.

[nmentley]

I've submitted the patch with the corrections you've suggested.

Just a few notes.

gcc/gcc/config/i386/haiku64.h: I irresponsibly left the comment "If ELF is the default format, we should not use /lib/elf" in from the file I based haiku64.h off of... Which happens to be haiku.h of the same directory. That comment is actually included in all the gcc/gcc/config/(arch)/haiku.h files strangely enough.

build/jam/BuildSetup: I'm currently reading up on the atomic_*() functions... So I might be alittle off here. In headers/os/support/SupportDefs.h: I've implemented sub, xor, nand, but I'm unsure of the atomic_set function. I've commented it out for now. I've included the newly defined atomic functions in the definition B_USE_EXTRA_BUILTIN_ATOMIC_FUNCTIONS since these builtins aren't supported in x86. This might not be the cleanest way to define the atomic functions tho.

headers/posix/arch/x86_64/arch_setjmp.h: I used 12 since it's the jmpbuf size on freebsd amd64, but that'll be changed to a more logical jmpbuf size later. Correcnt me if I'm wrong, but I was under the impression that setjmp() and longjmp() are defined in src/system/libroot/ponix/arch/(arch)/siglongjmp.S and src/system/libroot/ponix/arch/(arch)/sigsetjmp.S and not in this file. They don't have much to do with the gnu toolchain which was atleast my goal for this patch, but I'll for sure work on setjmp() and longjmp() before continuing working with haiku_loader.

Sorry about this patch being a bit shaky and thanks for the help.

[mmadia]

I added some text on how to enable "Automatic whitespace cleanup" : ​http://dev.haiku-os.org/wiki/SubmittingPatches#Automaticwhitespacecleanup

... only recently I learned about it & now I love it.

[stippi]

If I understood Ingo correctly, we don't want automatic white space cleanup for the buildtools files, since that means the patches grow bigger and have more chance of merge conflicts later.

[nmentley]

added a few header files in haiku/headers/posix/arch that'll allow for an x86_64 haiku target to be compiled. Also edited ./configure to support x86_64 arch when running build-cross-tools-gcc4. fenv.h is src/lib/msun/amd64/fenv.h from freebsd 8.0 file version: 1.6.10.1.2.1 2009/10/25 01:10:29

[bonefish]

Replying to nmentley:

Sorry for the delay. Things were a bit busy in the last week.

The latest Haiku trunk patch is somewhat broken. The third hunk for build/jam/BuildSetup has the wrong length, which causes patch to fail (with an unfortunately not very helpful error message). In case you edited the patch by hand, this is apparently something one shouldn't forget about.

I've submitted the patch with the corrections you've suggested.

Just a few notes.

gcc/gcc/config/i386/haiku64.h: I irresponsibly left the comment "If ELF is the default format, we should not use /lib/elf" in from the file I based haiku64.h off of... Which happens to be haiku.h of the same directory. That comment is actually included in all the gcc/gcc/config/(arch)/haiku.h files strangely enough.

Very irresponsibly indeed. ;-) Seems the comment originates from beos-elf.h already.

build/jam/BuildSetup: I'm currently reading up on the atomic_*() functions... So I might be alittle off here. In headers/os/support/SupportDefs.h: I've implemented sub, xor, nand, but I'm unsure of the atomic_set function. I've commented it out for now. I've included the newly defined atomic functions in the definition B_USE_EXTRA_BUILTIN_ATOMIC_FUNCTIONS since these builtins aren't supported in x86. This might not be the cleanest way to define the atomic functions tho.

There are no atomic_{sub,xor,nand}() in the public API, so there's no need to add those. Please note that the macros in <SupportDefs.h> are only there for optimization purposes. It doesn't matter whether macros are missing there (e.g. atomic_set()). The functions have to be implemented anyway (in src/system/libroot/os/arch/x86_64). Feel free to use hand-coded assembly or C++ just using the built-ins.

headers/posix/arch/x86_64/arch_setjmp.h: I used 12 since it's the jmpbuf size on freebsd amd64, but that'll be changed to a more logical jmpbuf size later.

OK, I was merely wondering because that is the same size as for x86, although the registers are twice as wide.

Correcnt me if I'm wrong, but I was under the impression that setjmp() and longjmp() are defined in src/system/libroot/ponix/arch/(arch)/siglongjmp.S and src/system/libroot/ponix/arch/(arch)/sigsetjmp.S and not in this file.

Yep, they are. You wouldn't get very far implementing them with a too short structure definition, though.

They don't have much to do with the gnu toolchain which was atleast my goal for this patch, but I'll for sure work on setjmp() and longjmp() before continuing working with haiku_loader.

OK.

Applied the buildtools patch in hrev36793. Applied the Haiku trunk patch in hrev36794, removing the extra atomic_*() part. Thanks a lot!

[anevilyak]

(In #9582) Indeed, forgot about that one.

[umccullough]

Other than that there are likely still bugs and some userland bits that aren't ported yet, should this ticket be closed now?

[xyzzy]

It should.