Debugger crashes when loading debuginfo

[KapiX]

Repro package: ​http://haiku.kacperkasper.pl/instdir.zip (356 MB)

1. cd program
2. LIBRARY_PATH=.:$LIBRARY_PATH svdemo
3. svdemo crashes. Click Debug.
4. Debugger will crash while loading debuginfo after some time.

1. export LIBRARY_PATH=.:$LIBRARY_PATH
2. LD_PRELOAD=libroot_debug.so MALLOC_DEBUG=g Debugger svdemo
3. svdemo crashes. Click Debug.
4. Debugger crashes almost immediately.

hrev50794 gcc5h

Reports attached.

[mmlr]

Replying to KapiX:

1. export LIBRARY_PATH=.:$LIBRARY_PATH
2. LD_PRELOAD=libroot_debug.so MALLOC_DEBUG=g Debugger svdemo
3. svdemo crashes. Click Debug.
4. Debugger crashes almost immediately.

The report with libroot_debug shows a debugger call due to failure to allocate a new memory area (i.e. the system's out of memory or the application ran out of address space). Considering that using libroot_debug with the guarded heap consumes a lot of extra memory, this isn't too surprising.

It is possible that the original crash was also due to an out of memory/address space situation since both stack traces seem to be in the path of hoard resizing. The handling of such situations should then be reviewed.

To avoid running out of address space you could try reproducing on a 64 bit installation instead. If the used memory indication in the report is correct, available memory shouldn't be the problem.

[anevilyak]

I was going to ask the same, this looks nearly identical to what occurs when attempting to debug WebKit on 32-bit, where we exhaust the address space while parsing the info. It may or may not also help to specify -fdebug-types-section in your debug build flags for your app.

[KapiX]

The issue with trying 64-bit is that it requires some more work without getting actual results. LibreOffice needs a long list of libraries present to even start compiling, and then the compilation takes around 6 hours on my machine.

I'll try stripping and keeping two sets of libraries - with and without debuginfo - and use debuginfo from one at a time. Maybe that will get me somewhere.

This should be fixed nonetheless because I don't think one failed allocation should bring the whole process down.

Bonus question: why do I see memory exhaustion when not even 50% of total available memory is allocated? I know there are limits of allocated memory for one process, but isn't it around 2 GB mark?

[anevilyak]

It's more complicated than that: the userland of a single process has 2gb of address space to work with. Note that this is not entirely equivalent to having 2gb of memory to work with since, e.g. the area mappings for the various shared libraries the app uses take up address space despite not taking additional memory for the code segments that are common across apps. In addition, there are other area mappings that are needed for things like system calls, which again take up address space without taking up memory. All of this also potentially results in address space fragmentation such that an area allocation may fail simply due to there not being a large enough contiguous chunk to satisfy the request. The latter may be what's happening with hoard, but at the moment it's difficult to tell for sure.

-fdebug-types-section would be worth trying on 32-bit either way, since it allows gcc to eliminate a lot of redundant debug info, which may lower the memory requirements (this option is disabled by default since not all versions of various open source dev tools know how to grok .debug_types).

[anevilyak]

Moving to libroot since this appears to be an issue with either hoard itself, or our backend. The codepaths in question on the debugger side use std::nothrow and check the allocation result, so the expected behavior would be for the allocation to fail and return NULL.

[waddlesplash]

Debugger doesn't crash while loading the debuginfo anymore, it seems the new malloc has fixed this one too. In GUI mode it seems to get stuck in some kind of infinite loop building the stack trace (or at least it was multiple minutes in there and didn't complete before I killed it), and in "Save report" mode it crashes with what looks like probably a NULL dereference in BString. That sounds like a different problem, and this ticket can be closed.

[waddlesplash]

I fixed the BString-related crashes in hrev58789. Debugger now crashes like this:

	thread 2143: DebugReportGenerator 
		state: Exception (Segment violation)

		Frame		IP			Function Name
		-----------------------------------------------
		0x702e7720	0x838b9f	DebugReportGenerator::_ResolveValueIfNeeded(ValueNode*, StackFrame*, int32) + 0x1f 
			Disassembly:
				DebugReportGenerator::_ResolveValueIfNeeded(ValueNode*, StackFrame*, int32):
				0x00838b80:               55  push %ebp
				0x00838b81:             89e5  mov %esp, %ebp
				0x00838b83:           83ec1c  sub $0x1c, %esp
				0x00838b86:               57  push %edi
				0x00838b87:               56  push %esi
				0x00838b88:               53  push %ebx
				0x00838b89:       e800000000  call 0x838b8e
				0x00838b8e:               5b  pop %ebx
				0x00838b8f:     81c382f81700  add $0x17f882, %ebx
				0x00838b95:           8b7d0c  movl 0xc(%ebp), %edi
				0x00838b98:   c745f400000000  movl $0x0, -0xc(%ebp)
				0x00838b9f:         837f1801  cmpl $0x1, 0x18(%edi) <--

			Frame memory:
				[0x702e76f0]  .......Q..?.....   10 84 9b 00 c0 13 8d 51 e0 10 3f 04 1d 8f 82 01
				[0x702e7700]  .......Q.y.p....   10 84 9b 00 c0 13 8d 51 98 79 2e 70 00 00 00 00
				[0x702e7710]  ..?......y.p.w..   e0 10 3f 04 05 00 00 00 98 79 2e 70 90 77 83 00
		0x702e79a0	0x83778b	DebugReportGenerator::_DumpDebuggedThreadInfo(BFile&, Thread*) + 0x8a3 
		0x702e7a10	0x836cd9	DebugReportGenerator::_DumpRunningThreads(BFile&) + 0x365 
		0x702e7ab0	0x834ea8	DebugReportGenerator::_GenerateReport(char*) + 0x80 
		0x702e7b30	0x8350b4	DebugReportGenerator::MessageReceived(BMessage*) + 0x68 
		0x702e7b60	0x16a6149	BLooper::DispatchMessage(BMessage*, BHandler*) + 0x59 
		0x702e7ba0	0x16a7eb3	BLooper::task_looper() + 0x203 
		0x702e7bd0	0x16a7aa5	BLooper::_task0_(void*) + 0x3d 
		0x702e7bf8	0x236ba4b	thread_entry + 0x27 
		00000000	0x618e7258	commpage_thread_exit + 0 

		Registers:
			  eip:	0x00838b9f
			  esp:	0x702e76f0
			  ebp:	0x702e7718
			  eax:	0x00000000
			  ebx:	0x009b8410
			  ecx:	0x04155700
			  edx:	0x7da80140
			  esi:	0x518d13c0
			  edi:	0x00000000
			   cs:	0x001b
			   ds:	0x0023
			   es:	0x0023
			   fs:	0x0063
			   gs:	0x0023
			   ss:	0x0023

So another NULL dereference presumably due to OOM. Probably there are more of these lurking, if there's really interest in fixing them we could open new tickets.