application crash - assert_fail in malloc

[andrewz]

Crash is repeatable when code uses 4 threads, but appears to run when 1 thread is used. Crash appears to be in Haiku malloc(), not in application code.

Reproduceable crash with scene 3

Debug information for team /boot/home/downloads/BeTracer (6293):
CPU(s): 4x Intel Core i5
Memory: 3.80 GiB total, 406.92 MiB used
Haiku revision: hrev49974 Jan 3 2016 18:43:35 (BePC)

Active Threads:
thread 6293: BeTracer (main)
thread 6297: w>BeTracer
thread 6311: name
thread 6312: name
thread 6313: name
thread 6315: team 6293 debug task
thread 6314: name
state: Call (getNumAvailable() == 0)

Frame	IP	Function Name
0x7983f368	0x812a31	debugger + 0x31
0x7983f388	0x820193	__assert_fail + 0x43
0x7983f3a8	0x8118cf	/boot/system/lib/x86/libroot.so + 0x268cf
0x7983f3e0	0x898a26	BPrivate::hoardHeap::findAvailableSuperblock(int, BPrivate::block*&, BPrivate::processHeap*) + 0x416
0x7983f430	0x896b10	BPrivate::threadHeap::malloc(unsigned long) + 0x70
0x7983f470	0x896e47	malloc + 0xa7
0x7983f490	0x11bda60	operator new(unsigned long) + 0x20
0x7983f4e0	0xb3aa46	CSG::CSGIntersection::texture() + 0xd6
0x7983f530	0xb3a990	CSG::CSGIntersection::texture() + 0x20
0x7983f5a0	0xb4684b	LightRay::shoot(float&) + 0x11b
0x7983f630	0xb488e6	Texture::enlightLight(LightRay&, CameraRay const&) const + 0x36
0x7983f6c0	0xb4a2ed	PointLight::enlight(CameraRay const&, Texture const*, InsideList*) const + 0x4d
0x7983f7c0	0xb45f8f	Scene::intersectRec(CameraRay&) + 0x12f
0x7983f7f0	0xb4648c	Scene::intersect(CameraRay&) + 0x2c
0x7983f890	0xb399cf	CameraPerspective::evaluateAt(Vector3 const&) const + 0x12f
0x7983f940	0xb44852	RenderThread::Run() + 0x1a2
0x7983f960	0xb44c3c	ThreadPrimitive::RunBinder(void*) + 0xc
0x7983f988	0x818e79	thread_entry + 0x19
00000000	0x60d70250	commpage_thread_exit + 0

[jua]

Hitting such assertions in the allocator usually point to heap corruption in the application. While a Haiku bug is possible, it's most likely an application bug, e.g. an out of bounds array write. Try to use libroot_debug which should help in finding the source of the problem.

[mmlr]

Specifically running the application like this:

LD_PRELOAD=libroot_debug.so MALLOC_DEBUG=g <application>

This will align allocations so that they are as close as possible to unmapped pages which makes it likely for the application to crash on the actual invalid access.

As jua says the assert is triggered by the allocator because it detected that its data structures were corrupted. The actual cause is almost always an application error causing such corruption.