Opened 11 years ago

Last modified 2 years ago

#3127 assigned bug

NULL pointer access inside fork()

Reported by: bhaible Owned by: nobody
Priority: normal Milestone: R1
Component: - General Version: R1/pre-alpha1
Keywords: Cc:
Blocked By: Blocking:
Has a Patch: no Platform: All

Description

While running a configure script in tight memory situation, I got an application crash. Clicking on the "Debug" button of the error dialog, I was presented a gdb session the shell that was executing the configure script. At the top, function fork().

To reproduce:

  1. Start Haiku in QEMU with 256 MB RAM allocated to it,
  1. Open a Terminal.
  1. $ wget http://www.haible.de/bruno/gnu/testdir3.tar.gz
  1. $ tar xvfz testdir3.tar.gz
  1. $ cd testdir3
  1. $ ./configure --host=i586-pc-haiku --build=i586-pc-haiku

--prefix=/boot/home/config CPPFLAGS="-Wall" 2>&1 | tee log1

  1. Wait a couple of hours.

Change History (6)

comment:1 Changed 11 years ago by axeld

I would guess that's not too surprising - if it runs out of memory there is little it can do. But I keep it open to have a deeper look at it first.

comment:2 Changed 11 years ago by bonefish

The general memory tightness is apparently caused by a kernel memory leak (cf. #3128). Generally fork() should report ENOMEM instead of crashing, but if the virtual memory is really exhausted, touching previously unused stack will definitely crash programs, since the stacks are over-committing. And there's little else we can do in such a case. The syslog (/var/log/syslog) should help to verify whether the page fault happened on the stack.

comment:3 Changed 11 years ago by axeld

We could let fork() reserve one or two stack pages for the new team, so that it could fail with ENOMEM in most of these cases as well.

comment:4 in reply to:  3 ; Changed 11 years ago by bonefish

Replying to axeld:

We could let fork() reserve one or two stack pages for the new team, so that it could fail with ENOMEM in most of these cases as well.

That doesn't make much sense. fork() is not supposed to return an error code in the child. Crashing really is the only option I'm afraid.

comment:5 in reply to:  4 Changed 11 years ago by axeld

Replying to bonefish:

That doesn't make much sense. fork() is not supposed to return an error code in the child. Crashing really is the only option I'm afraid.

Who said it would do that? The areas of the new team are allocated in the parent, so we can also commit memory there for the child; if that fails, we can let fork() fail which would cause it to return ENOMEM in the parent process, before the child has been started.

comment:6 Changed 2 years ago by axeld

Owner: changed from axeld to nobody
Status: newassigned
Note: See TracTickets for help on using tickets.