Opened 7 years ago

Last modified 7 years ago

#8761 assigned bug

Software renderer should gracefully handle unsupported color spaces.

Reported by: xray7224 Owned by: kallisti5
Priority: normal Milestone: R1
Component: Kits/OpenGL Kit Version: R1/Development
Keywords: Cc:
Blocked By: Blocking:
Has a Patch: no Platform: All

Description (last modified by anevilyak)

Currently, if one attempts to instantiate the mesa software renderer (legacy or new) in an unsupported color space (i.e. if the desktop is set to 8-bit color), the app will crash into the debugger. This should be handled a bit more gracefully. Simply removing the debugger() call doesn't appear to suffice though, since the return from _SetupRenderBuffer() is ignored, causing issues elsewhere later.

Change History (5)

comment:1 Changed 7 years ago by anevilyak

Are you running in 8-bit color mode? If so, that's not currently supported.

comment:2 Changed 7 years ago by xray7224

I was yes, sorry. I don't know why it defaults to that. It works fine now i've switched it to 32 bit mode.

comment:3 Changed 7 years ago by anevilyak

Description: modified (diff)
Owner: changed from korli to kallisti5
Status: newassigned
Summary: OpenGL Kits crashing.Software renderer should gracefully handle unsupported color spaces.

No problem, it should fail gracefully in that case instead of simply hitting the debugger though.

comment:4 Changed 7 years ago by kallisti5

Yeah, I agree... I think it does try to return gracefully though on 8-bit color: http://cgit.haiku-os.org/haiku/tree/src/add-ons/opengl/swrast/MesaSoftwareRenderer.cpp#n608

We try to handle _SetupRenderBuffer failures gracefully here: http://cgit.haiku-os.org/haiku/tree/src/add-ons/opengl/swrast/MesaSoftwareRenderer.cpp#n599

However here, not so much... http://cgit.haiku-os.org/haiku/tree/src/add-ons/opengl/swrast/MesaSoftwareRenderer.cpp#n235

I probably should check color spaces much earlier on in the renderer. Any changes made to swrast need to made identically as possible in swrast_legacy.

I'll take a crack at it :)

comment:5 Changed 7 years ago by kallisti5

hmm.. actually we do try to bail on incorrect colorspace here: http://cgit.haiku-os.org/haiku/tree/src/add-ons/opengl/swrast/MesaSoftwareRenderer.cpp#n151

either way, I'll fix it. Probably something getting dereferenced or something.

Note: See TracTickets for help on using tickets.