Mesa Software Renderer crash with SSE optimization enabled

[phoudoin]

When libGL.so and Mesa Software Renderer are built with USE_SSE_ASM defined to enable Mesa's SSE optimization, GLDirectMode crash at start and GLTeapot crash after a short time. Only GLInfo don't crash, but it's expected as it does no GL drawing at all.

The crash is due to an invalid opcode exception:

Thread 334 caused an exception: Invalid opcode exception

[...]

[Switching to team /boot/beos/demos/GLDirectMode (334) thread GLDirectMode (334)]
0x00846e20 in _mesa_sse_transform_points3_general ()
   from /boot/beos/system/add-ons/opengl/Mesa Software Renderer

Disassembled code show the opcode is movaps (%edx),%xmm0

I'm not fluent in SSE instruction set enough to dive in more. Anyone know if and where to find some MMX/SSE/SSE2/SSE3/SSE4 stress tests that could be run under Haiku?

[phoudoin]

Complete GDB log and faulty code disassembly:

GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i586-pc-haiku"...(no debugging symbols found)

[tcsetpgrp failed in terminal_inferior: Invalid Argument]
Thread 334 caused an exception: Invalid opcode exception
Reading symbols from /boot/beos/system/runtime_loader...done.
Loaded symbols for /boot/beos/system/runtime_loader
Reading symbols from /boot/beos/system/lib/libbe.so...done.
Loaded symbols for /boot/beos/system/lib/libbe.so
Reading symbols from /boot/beos/system/lib/libgame.so...done.
Loaded symbols for /boot/beos/system/lib/libgame.so
Reading symbols from /boot/beos/system/lib/libGL.so...done.
Loaded symbols for /boot/beos/system/lib/libGL.so
Reading symbols from /boot/beos/system/lib/libroot.so...done.
Loaded symbols for /boot/beos/system/lib/libroot.so
Reading symbols from /boot/beos/system/lib/libstdc++.r4.so...done.
Loaded symbols for /boot/beos/system/lib/libstdc++.r4.so
Reading symbols from /boot/beos/system/lib/libmedia.so...done.
Loaded symbols for /boot/beos/system/lib/libmedia.so
Reading symbols from /boot/beos/system/add-ons/opengl/Mesa Software Renderer...done.
Loaded symbols for /boot/beos/system/add-ons/opengl/Mesa Software Renderer
[tcsetpgrp failed in terminal_inferior: Invalid Argument]
[Switching to team /boot/beos/demos/GLDirectMode (334) thread GLDirectMode (334)]
0x00846e20 in _mesa_sse_transform_points3_general ()
   from /boot/beos/system/add-ons/opengl/Mesa Software Renderer
(gdb) disassemble
Dump of assembler code for function _mesa_sse_transform_points3_general:
0x00846de0 <_mesa_sse_transform_points3_general+0>:     push   %esi
0x00846de1 <_mesa_sse_transform_points3_general+1>:     push   %edi
0x00846de2 <_mesa_sse_transform_points3_general+2>:     mov    0x14(%esp),%esi
0x00846de6 <_mesa_sse_transform_points3_general+6>:     mov    0xc(%esp),%edi
0x00846dea <_mesa_sse_transform_points3_general+10>:    mov    0x10(%esp),%edx
0x00846dee <_mesa_sse_transform_points3_general+14>:    mov    0x8(%esi),%ecx
0x00846df1 <_mesa_sse_transform_points3_general+17>:    cmp    $0x0,%ecx
0x00846df4 <_mesa_sse_transform_points3_general+20>:    je     0x846e78 <_mesa_sse_transform_points3_general+152>
0x00846dfa <_mesa_sse_transform_points3_general+26>:    mov    0xc(%esi),%eax
0x00846dfd <_mesa_sse_transform_points3_general+29>:    orl    $0xf,0x14(%edi)
0x00846e01 <_mesa_sse_transform_points3_general+33>:    mov    %ecx,0x8(%edi)
0x00846e04 <_mesa_sse_transform_points3_general+36>:    movl   $0x4,0x10(%edi)
0x00846e0b <_mesa_sse_transform_points3_general+43>:    shl    $0x4,%ecx
0x00846e0e <_mesa_sse_transform_points3_general+46>:    mov    0x4(%esi),%esi
0x00846e11 <_mesa_sse_transform_points3_general+49>:    mov    0x4(%edi),%edi
0x00846e14 <_mesa_sse_transform_points3_general+52>:    add    %edi,%ecx
0x00846e16 <_mesa_sse_transform_points3_general+54>:    lea    0x0(%esi),%esi
0x00846e19 <_mesa_sse_transform_points3_general+57>:    lea    0x0(%edi),%edi
0x00846e20 <_mesa_sse_transform_points3_general+64>:    movaps (%edx),%xmm0
0x00846e23 <_mesa_sse_transform_points3_general+67>:    movaps 0x10(%edx),%xmm1
0x00846e27 <_mesa_sse_transform_points3_general+71>:    movaps 0x20(%edx),%xmm2
0x00846e2b <_mesa_sse_transform_points3_general+75>:    movaps 0x30(%edx),%xmm3
---Type <return> to continue, or q <return> to quit---
0x00846e2f <_mesa_sse_transform_points3_general+79>:    nop    
0x00846e30 <_mesa_sse_transform_points3_general+80>:    nop    
0x00846e31 <_mesa_sse_transform_points3_general+81>:    nop    
0x00846e32 <_mesa_sse_transform_points3_general+82>:    nop    
0x00846e33 <_mesa_sse_transform_points3_general+83>:    nop    
0x00846e34 <_mesa_sse_transform_points3_general+84>:    nop    
0x00846e35 <_mesa_sse_transform_points3_general+85>:    nop    
0x00846e36 <_mesa_sse_transform_points3_general+86>:    nop    
0x00846e37 <_mesa_sse_transform_points3_general+87>:    nop    
0x00846e38 <_mesa_sse_transform_points3_general+88>:    nop    
0x00846e39 <_mesa_sse_transform_points3_general+89>:    nop    
0x00846e3a <_mesa_sse_transform_points3_general+90>:    nop    
0x00846e3b <_mesa_sse_transform_points3_general+91>:    nop    
0x00846e3c <_mesa_sse_transform_points3_general+92>:    nop    
0x00846e3d <_mesa_sse_transform_points3_general+93>:    nop    
0x00846e3e <_mesa_sse_transform_points3_general+94>:    nop    
0x00846e3f <_mesa_sse_transform_points3_general+95>:    nop    
0x00846e40 <_mesa_sse_transform_points3_general+96>:    movss  (%esi),%xmm4
0x00846e44 <_mesa_sse_transform_points3_general+100>:   shufps $0x0,%xmm4,%xmm4
0x00846e48 <_mesa_sse_transform_points3_general+104>:   movss  0x4(%esi),%xmm5
0x00846e4d <_mesa_sse_transform_points3_general+109>:   shufps $0x0,%xmm5,%xmm5
0x00846e51 <_mesa_sse_transform_points3_general+113>:   movss  0x8(%esi),%xmm6
0x00846e56 <_mesa_sse_transform_points3_general+118>:   shufps $0x0,%xmm6,%xmm6
0x00846e5a <_mesa_sse_transform_points3_general+122>:   mulps  %xmm0,%xmm4
---Type <return> to continue, or q <return> to quit---

[korli]

I tried to dump cr0 et cr4 on invalid code exception, and in fact cr4 is equal to zero. This is a condition for the exception to be thrown on SSE 128bits instructions. I don't get why and where this register is cleared though.

[korli]

This should be fixed in hrev29053.