[Gravity] crashes in BBitmap::_AssertPointer()

[diver]

hrev46759, gcc2 hybrid.

I was trying to repro #10209 when Haiku crashed. Upon reboot subsequent attempts to start ScreenSaver resulted in this crash. After deleting ScreenSaver settings file everything went back to normal.

state: Exception (Segment violation)
Frame		IP		Function Name
		-----------------------------------------------
0x70ff2ce8	0x19b7647	BBitmap::_AssertPointer() + 0x17
[...]
0x70ff2d18	0x19b64df	BBitmap::Bits(BBitmap) + 0x1b
0x70ff2d78	0xd4e12e	MesaSoftwareRenderer::_AllocateBitmap() + 0x116
0x70ff2da8	0xd4f947	MesaSoftwareRenderer::_CheckResize(uint32, uint32) + 0x5b
0x70ff2df8	0xd4d788	MesaSoftwareRenderer::LockGL() + 0x100
0x70ff3108	0xd4d5e6	__20MesaSoftwareRendererP7BGLViewUlP13BGLDispatcher + 0x4a2
0x70ff3148	0xd4f619	instantiate_gl_renderer + 0x31
0x70ff3208	0x14ac382	GLRendererRoster::CreateRenderer(entry_ref&) + 0x10a
0x70ff32f8	0x14abde0	GLRendererRoster::AddPath(char*) + 0x178
0x70ff3368	0x14abbfe	GLRendererRoster::AddDefaultPaths() + 0xa2
0x70ff35b8	0x14ab9e6	__16GLRendererRosterP7BGLViewUl + 0x262
0x70ff3608	0x14a9ff3	__7BGLViewG5BRectPCcUlUlUl + 0x117
0x70ff3668	0x1c0b2a4	__11GravityViewP7GravityG5BRect + 0x4c
0x70ff36b8	0x1c0aea2	Gravity::StartSaver(BView*, bool) + 0x72
0x70ff3728	0x14dc24b	ModulesView::_OpenSaver() + 0x20b
0x70ff37c8	0x14db497	ModulesView::MessageReceived(BMessage*) + 0x117
0x70ff37f8	0x198d379	BLooper::DispatchMessage(BMessage*, BHandler*) + 0x59
0x70ff39f8	0x1a697cc	BWindow::DispatchMessage(BMessage*, BHandler*) + 0x1970
0x70ff3a58	0x1a6dbf2	BWindow::task_looper() + 0x28e
0x70ff3a88	0x198e811	BLooper::_task0_(void*) + 0x3d
0x70ff3ab0	0x17a5681	thread_entry + 0x21
00000000	0x60c94250	commpage_thread_exit + 0

[diver]

Reproducible with the attached settings file.

[kallisti5]

Hm, this is actually one of the more interesting backtraces I've seen for this issue. It appears that Mesa 7.9.2 crashes while allocating the BBitmap for drawing, and Mesa 10.0.2 swrast crashes in the same spot. Mesa 10.0.2 swpipe crashes in winsys which is where our gallium renderer allocates a BBitmap. I bet flurry is somehow passing an incorrect bitmap size to OpenGL causing an assertions when it attempts to write beyond the bitmap bits length.

I think it sounds like Mesa needs better checking of bitmap sizes before writing to the bits, and flurry has some bug that exercises this issue.

[pulkomandy]

I tried to debug mesa_legacy, and I found that there is an integer overflow. The fWidth and fHeight used when allocating the bitmap are unsigned ints, so an attempt to set them to -1 leads to an overflow. This prevents the already existing "<1" checks from working. Since we also don't check wether the bitmap allocation and InitCheck succeded, we hit a BBitmap assert when accessing the non existing bits.

[pulkomandy]

Fixed in hrev48011 and hrev48012.