ARM loader: arch_mmu positioning dynamic va start incorrectly.

[kallisti5]

src/system/boot/arch/arm/arch_mmu.cpp

// Mark start for dynamic allocation
IsNextPhysicalAddress =
IsNextVirtualAddress = sPageTableRegionEnd;

(per mmu_man)

src/system/boot/arch/arm/arch_mmu.cpp:	sPageTableRegionEnd = (addr_t)sPageDirectory + 0x200000;

src/system/boot/arch/arm/arch_mmu.cpp:	sPageDirectory = (uint32 *)ROUNDUP((addr_t)&_end, 0x100000);


src/system/boot/arch/arm/arch_mmu.cpp:extern int _start, _end;


./src/system/ldscripts/arm/boot_loader_u-boot.ld
      . = BOARD_LOADER_BASE;
      .
      .
      _end = . ;

Example boot on Raspberry Pi 2..

reading /boot.scr
312 bytes read in 15 ms (19.5 KiB/s)
## Executing script at 00000000
reading bcm2836-rpi-2-b.dtb
5690 bytes read in 16 ms (346.7 KiB/s)
reading haiku-floppyboot.tgz.ub
1596766 bytes read in 600 ms (2.5 MiB/s)
reading haiku_loader_linux.ub
288696 bytes read in 124 ms (2.2 MiB/s)
## Booting kernel from Legacy Image at 01000000 ...
   Image Name:   haiku_loader rpi2
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    288632 Bytes = 281.9 KiB
   Load Address: 00080000
   Entry Point:  00080010
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 02100000 ...
   Image Name:   haiku-floppyboot.tgz rpi2
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    1596702 Bytes = 1.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02000000
   Booting using the fdt blob at 0x2000000
   Loading Kernel Image ... OK
   Loading Ramdisk to 079c4000, end 07b49d1e ... OK
   Loading Device Tree to 079bf000, end 079c3639 ... OK

Starting kernel ...

�check_cpu_features: implementor=0x41('A'), arch=9, variant=0x0, part=0xc07, revision=0x5
Found boot tgz from FDT @ 0x079c4000, 1596702 bytes
argc = 0
os: 2
gd @ 0x00000000
FDT @ 0x079bf000:
fdt_totalsize: 8320
fdt_off_dt_struct: 88
fdt_off_dt_strings: 5252
fdt_off_mem_rsvmap: 40
fdt_version: 17
fdt_last_comp_version: 16
fdt_boot_cpuid_phys: 0
fdt_size_dt_strings: 733
fdt_size_dt_struct: 5164
checking for memory...
0: base = 0,size = 134217728
total physical memory = 128MB
*** PANIC ***
*** PANIC ***
map_page: asked to map invalid page 0x00300000!
map_page: asked to map invalid page 0x00300000!


Press key to reboot.
*** PANIC ***
*** PANIC ***
map_page: asked to map invalid page 0x00301000!
map_page: asked to map invalid page 0x00301000!


Press key to reboot.
*** PANIC ***
*** PANIC ***
map_page: asked to map invalid page 0x00302000!
map_page: asked to map invalid page 0x00302000!


Press key to reboot.
.
.

[pulkomandy]

I'm not sure about that, the mmu code was written to work with several different mappings and the beagleboard was the first time we had RAM at this address, which was a problem because the mmu code wouldn't handle it too well.

Some details about the MMU mapping process:

• We start with the MMU disabled. u-boot loads things at the relevant physical addresses
• The bootloader setups an identity mapping, that is, relevant parts of the RAM (where the bootloader and kernel code are loaded, plus the bootloader stack, heap), and the IO devices are mapped so their virtual addresses matches their physical addresses
• The MMU is enabled, since all the useful parts are identity mapped this is transparent and the bootloader code can continue to execute,
• The kernel is remapped to it's final target (virtual addresses starting at 0x8000000, physical address unchanged), and its stack and a small part of its heap are mapped in that area as well.
• The kernel is booted
• The kernel MMU initialization removes the now useless identity mapping.

So the 80000000 you are seeing is setup from the boot loader, and is not related to the RAM physical address in any way.

[kallisti5]

The 0x00300000 it is attempting to map seems to be the physical address for the MMC card. (​http://cgit.haiku-os.org/haiku/tree/headers/private/kernel/arch/arm/bcm283X.h#n80)

I'm guessing u-boot handed the MMC pointer off? One odd thing is that address should be 0x3f300000 as it is BCM283X_PERIPHERAL_BASE + EMMC_BASE.

I wonder if that's a u-boot bug?

[pulkomandy]

Have you enabled TRACE in the mmu files? It should log more info about what it does, which would help finding the problem.

[kallisti5]

Good call, forgot about that :-)

Attached.

[kallisti5]

mmu debug. some lines truncated

[pulkomandy]

75	mmu_map_identity: [ 100000 - 300000]
76	get_next_page_table, sNextPageTableAddress 0x104400, sPageTableRegionEnd 0x300000, type 0x1
77	get_next_page_table, sNextPageTableAddress 0x104800, sPageTableRegionEnd 0x300000, type 0x1

Then a bit later:

107	map_page: vaddr 0x300000, paddr 0x300000
108	*** PANIC ***

So it would seem the initial identity mapping for that area is not big enough, and we try to access something at address 0x300000 which was not properly mapped?

[kallisti5]

Ah. I missed that fact :-)

kallisti5@eris haiku :) $ grep -RnA1 "mmu_map_identity" src/

system/boot/arch/arm/arch_mmu.cpp:284:mmu_map_identity(addr_t start, size_t end, int flags)
--
system/boot/arch/arm/arch_mmu.cpp:292:TRACE(("mmu_map_identity: [ %" B_PRIxADDR " - %" B_PRIxADDR "]\n", start, end));
--
system/boot/arch/arm/arch_mmu.cpp:320:mmu_map_identity((addr_t)&_start, (addr_t)&_end, ARM_MMU_L2_FLAG_C);
--
system/boot/arch/arm/arch_mmu.cpp:323:mmu_map_identity((addr_t)sPageDirectory, sPageTableRegionEnd, ARM_MMU_L2_FLAG_C);
--
system/boot/arch/arm/arch_mmu.cpp:333:mmu_map_identity(LOADER_MEMORYMAP[i].start, LOADER_MEMORYMAP[i].end,
system/boot/arch/arm/arch_mmu.cpp-334-  LOADER_MEMORYMAP[i].flags);

[kallisti5]

Looking at the logs and the mmu code,

src/system/boot/arch/arm/arch_mmu.cpp

init_page_directory()...

    // ****
    // 80000 - d0000 here....
    // *****
    // map ourselfs first... just to make sure
    mmu_map_identity((addr_t)&_start, (addr_t)&_end, ARM_MMU_L2_FLAG_C);

    // ****
    // 100000 - 300000 here....
    // *****
    // map our page directory region (TODO should not be identity mapped)
    mmu_map_identity((addr_t)sPageDirectory, sPageTableRegionEnd, ARM_MMU_L2_FLAG_C);

So what is attempting to use the 300000+?

[kallisti5]

Actually, looking at mmu_man's comments... this makes sense.

dynamic allocation is at the PageTableRegionEnd...

    // allocate page directory in memory after loader
    sPageDirectory = (uint32 *)ROUNDUP((addr_t)&_end, 0x100000);
    sNextPageTableAddress = (addr_t)sPageDirectory + ARM_MMU_L1_TABLE_SIZE;
    sPageTableRegionEnd = (addr_t)sPageDirectory + 0x200000;

    // Mark start for dynamic allocation
    sNextPhysicalAddress =
    sNextVirtualAddress = sPageTableRegionEnd;

So:

• loader 80000 - d0000
  • insert_physical_allocated_range
• pageDirectory 100000 - 300000
  • insert_physical_allocated_range
• dynamicAllocation 300000+

• init_page_directory();
  • mmu_map_identity loader 80000 - d0000
  • mmu_map_identity pageDirectory 100000 - 300000

This all means the first map_page request for a dynamic range coming in fails..

mmu_allocate: requested vaddr: 0x00000000, next free vaddr: 0x300000, size: 16384
map_page: vaddr 0x300000, paddr 0x300000
*** PANIC ***
*** PANIC ***
map_page: asked to map invalid page 0x00300000!
map_page: asked to map invalid page 0x00300000!

mmu_allocate says the first virtual memory for the dynamic range is 0x300000... is that right?

static void
map_page(addr_t virtualAddress, addr_t physicalAddress, uint32 flags)
{
    TRACE(("map_page: vaddr 0x%lx, paddr 0x%lx\n", virtualAddress,
        physicalAddress));

    if (virtualAddress < KERNEL_LOAD_BASE) {
        panic("map_page: asked to map invalid page %p!\n",
            (void *)virtualAddress);
    }

headers/private/kernel/arch/arm/arch_kernel.h:#define KERNEL_BASE 0x80000000

so.... there is the issue. We're setting vaddr to start at 0x300000, but the mmu_page checks for >= KERNEL_BASE (0x80000000)

[kallisti5]

would it be enough to set sNextVirtualAddress to KERNEL_LOAD_BASE?

[kallisti5]

patch. I need to test this evening.

[kallisti5]

... although maybe sNextVirtualAddress should be KERNEL_LOAD_BASE + kMaxKernelSize?

[pulkomandy]

This address is the start of the dynamically allocated RAM. We are apparently trying to identity-map it (va==pa) but we also want it to be in kernel space (>= KERNEL_LOAD_BASE). This isn't possible with the Pi memory map.

So, KERNEL_LOAD_BASE + kMaxKernelSize would make sense. It makes sure we don't erase the kernel with allocated memory, and moves the virtual addresses to kernel-side address space.

[kallisti5]

patch v2 pushing the va start address past the kernel. Will test this evening.

[kallisti5]

That patch definitely makes things *much* better...

MMC:   bcm2835_sdhci: 0
reading uboot.env

** Unable to read "uboot.env" from mmc0:1 **
Using default environment

In:    serial
Out:   lcd                                                                                                                
Err:   lcd                                                                                                                
Net:   Net Initialization Skipped                                                                                         
No ethernet found.                                                                                                        
Hit any key to stop autoboot:  0                                                                                          
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
reading /boot.scr
312 bytes read in 15 ms (19.5 KiB/s)
## Executing script at 00000000
reading bcm2836-rpi-2-b.dtb
5690 bytes read in 16 ms (346.7 KiB/s)
reading haiku-floppyboot.tgz.ub
1596781 bytes read in 600 ms (2.5 MiB/s)
reading haiku_loader_linux.ub
288696 bytes read in 124 ms (2.2 MiB/s)
## Booting kernel from Legacy Image at 01000000 ...
   Image Name:   haiku_loader rpi2
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    288632 Bytes = 281.9 KiB
   Load Address: 00080000
   Entry Point:  00080010
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 02100000 ...
   Image Name:   haiku-floppyboot.tgz rpi2
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    1596717 Bytes = 1.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02000000
   Booting using the fdt blob at 0x2000000
   Loading Kernel Image ... OK
   Loading Ramdisk to 079c4000, end 07b49d2d ... OK
   Loading Device Tree to 079bf000, end 079c3639 ... OK

Starting kernel ...

�check_cpu_features: implementor=0x41('A'), arch=9, variant=0x0, part=0xc07, revision=0x5
Found boot tgz from FDT @ 0x079c4000, 1596717 bytes
argc = 0
os: 2
gd @ 0x00000000
FDT @ 0x079bf000:
fdt_totalsize: 8320
fdt_off_dt_struct: 88
fdt_off_dt_strings: 5252
fdt_off_mem_rsvmap: 40
fdt_version: 17
fdt_last_comp_version: 16
fdt_boot_cpuid_phys: 0
fdt_size_dt_strings: 733
fdt_size_dt_struct: 5164
checking for memory...
0: base = 0,size = 134217728
total physical memory = 128MB
Found bootargs: 
args.arguments_count = 1
args.arguments[0] @8098a1b0 = ''
adding args: ''
Welcome to the Haiku boot loader!
platform_add_boot_device
Memory Disk at: 0x80804000 size: 185d2d
add_partitions_for(0x8098d160, mountFS = no)
add_partitions_for(fd = 0, mountFS = no)
0x8098d1e0 Partition::Partition
0x8098d1e0 Partition::Scan()
check for partitioning_system: Intel Partition Map
check for partitioning_system: Intel Extended Partition

As long as mmu_man is cool with the change i'll commit it and close this one. (there is still a bug there though as it locks up at "check for partitioning_system: Intel Extended Partition"

[kallisti5]

full log + mmu debug post patch

[kallisti5]

hrev48898 fixed this issue for me on ARM on the Raspberry Pi, however it seems like it might have caused a regression on our qemu target overo. I'm trying to reproduce this now.

[pulkomandy]

Do we still care about the Overo target?

[richienyhus]

Replying to pulkomandy:

Do we still care about the Overo target?

It is an ARM Cortex-A8, so it can't hurt. Plus it looks like it has the same spec as the BeagleBoard-xM.

The verdex and neo-freerunner on the other hand are very much outdated...

[kallisti5]

I think Overo is fixed. There were some incorrect calculations in the FDT dev base functions a while back.

Something is wrong however with mmu. We're seeing strange behavior early on when decompressing the haiku-floppyroot.img.tgz into memory (crashing in empty constructors, using FATFS code instead of TARFS code, etc)

[kallisti5]

great news! ARM on beagle is in really good shape at the moment and can be emulated!

git clone //git.linaro.org/qemu/qemu-linaro.git
./configure
make -j8

Compile haiku arm:

mkdir generated.beagle; cd generated.beagle
../configure --build-cross-tools arm ../../buildtools --target-board beagle -j8
jam -q @minimum-mmc

Then run the emulator:

arm-softmmu/qemu-system-arm -machine beagle -m 512 -sd generated.beagle/haiku-beagle.mmc -serial stdio

You get full access to the KDL prompt over serial (stdio) once started :-)

[pulkomandy]

This is the same state the beagle port has been in since last year BeGeistert. How is that "news"?

Next step: add a mass storage driver, either write an SD/MMC stack, or adjust the USB drivers (only ehci needed?) so they do not depend on PCI and can use FDT instead. Then, continue booting!

[kallisti5]

News as I didn't know we had any arm boards that emulation worked successfully on :-)

[pulkomandy]

That was the main reason fpr picking the beagle-xm as our first target for the ARM port during last year GSoC (btw, you can set machine to beagle-xm IIRC, which will let you also see the boot screen and on-screen KDL. You still need a serial port to enter things into KDL however, as there is no keyboard driver yet).

[pulkomandy]

Can we close this ticket? I think the issue was solved already and the patch is not needed anymore?

[waddlesplash]

No reply, assuming fixed.