Possible floating point failures in IEEE 754 tests

[scottmc]

The python regression test suite is failing test_cmath:

test test_cmath failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/test/test_cmath.py", line 366, in test_specific_values
    self.fail(error_message)
AssertionError: acos0032: acos(complex(-0.99999999999999978, 0.0))
Expected: complex(3.141592638688631, -0.0)
Received: complex(3.1415926325163688, -0.0)
Received value insufficiently close to expected value.

test_decimal:

test test_decimal failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/test/test_decimal.py", line 1386, in test_from_float
    self.assertEqual(x, float(MyDecimal.from_float(x))) # roundtrip
AssertionError: 14.037217101164238 != 14.037217101164231

test_float:

test test_float failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/test/test_float.py", line 253, in test_repr
    self.assertEqual(v, eval(repr(v)))
AssertionError: 7.9999999872500254e+153 != 7.9999999745000492e+153

test_json:

test_json
test test_json failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/json/tests/test_float.py", line 9, in test_floats
    self.assertEquals(float(json.dumps(num)), num)
AssertionError: 5.1878483364637832e+49 != 5.1878483143195925e+49

test_matshall:

test test_marshal failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/test/test_marshal.py", line 95, in test_floats
    self.assertEqual(f, got)
AssertionError: -7.9456894939000011e-241 != inf

test_math:

test_math
test test_math failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/test/test_math.py", line 879, in test_testfile
    self.fail(message)
AssertionError: Unexpected ValueError in test acos0231:acos(inf)

[scottmc]

Also test_random:

test test_random failed -- Traceback (most recent call last):
  File "/boot/common/lib/python2.7/test/test_random.py", line 344, in test_referenceImplementation
    self.assertAlmostEqual(a,e,places=14)
AssertionError: 0.081823493762449573 != -inf within 14 places

[scottmc]

Here's a prebuilt binary of python-2.7-svn: ​http://www.fileden.com/files/2008/8/23/2062382/python-2.7-gcc2-2009-01-19-2009.zip regression tests are in /boot/common/lib/python2.7/test

To run just type: python test_name.py

[scottmc]

Here's a reply from Mark Dickinson on the python mailing list:

Most of these look like libm/libc precision problems to me,
of varying severity.  Some particular comments:

 - the test_float result is worrying: there are a good few places
  where Python depends on eval(repr(.)) round-tripping for
  floats, and it looks as though either the eval or the repr
  is losing significant accuracy.  Actually, there's so much
  accuracy loss that I wonder whether something's being
  cast from double precision to single precision at some
  point.

- test_decimal failing was a bit of a surprise until I saw
 which test was failing:  the decimal module quite
 deliberately does all computation using integer
 arithmetic, and avoids floating-point like the plague,
 so it should be ultra-portable.  Except, of course, the
 recently added from_float method, which converts
 from floats to decimals.  So fix up the floating-point
 and test_decimal should pass again.

- I don't understand where the test_marshall and
 test_random failures are coming from.  These
 could be Python problems (though I think it's
 more likely that they're Haiku floating-point
 problems).  I'd be interested to see short
 code-snippets that reproduce these issues.

- I wouldn't worry so much about the test_math
 and test_cmath failures until you get the others
 sorted out;  the tests are probably stricter than
 they need to be.

[scottmc]

Searching around for a math test I located this one:
​http://www.math.utah.edu/~beebe/software/ieee/

Compiled it and ran the tests, here's the results of most of them, some passing a few failing:

~/ieee> chkinexact
------------------------------------------------------------------------

Test at normal underflow limit

float: 2^(-126)
f = 1.17549e-38

double: 2^(-1022)
d = 2.22507e-308

LONG_DOUBLE: 2^(-16382)
q = 3.3621e-4932
------------------------------------------------------------------------

Test of divide-by-two at normal underflow limit

float: 2^(-127)
f = 5.87747e-39

double: 2^(-1023)
d = 1.11254e-308

LONG_DOUBLE: 2^(-16383)
q = 1.68105e-4932
------------------------------------------------------------------------

Test of divide-by-two at normal underflow limit

float: 2^(-128)
f = 2.93874e-39

double: 2^(-1024)
d = 5.56268e-309

LONG_DOUBLE: 2^(-16384)
q = 8.40526e-4933
------------------------------------------------------------------------

Test of subnormal underflow limit

float: 2^(-149)
f = 1.4013e-45

double: 2^(-1074)
d = 4.94066e-324

LONG_DOUBLE: 2^(-16494)
q = 0
------------------------------------------------------------------------

Test of divide-by-two at subnormal underflow limit

float: 2^(-150)
f = 0

double: 2^(-1075)
d = 0

LONG_DOUBLE: 2^(-16495)
q = 0
------------------------------------------------------------------------
~/ieee> copysign
-0 > +0: FALSE
-0 < +0: FALSE
-0 == +0: TRUE
-0 == +0: TRUE
copysign(0,-1) = -0.000000
copysign(0,1) = 0.000000
~/ieee> datasize
Pointer sizes...
        sizeof(char*)           = 4
        sizeof(void*)           = 4
        sizeof(void(*)(void))   = 4


Integer sizes...
        sizeof(char)            = 1
        sizeof(short)           = 2
        sizeof(int)             = 4
        sizeof(long)            = 4
        sizeof(long long)       = 8


Floating-point sizes...
        sizeof(float)           = 4
        sizeof(double)          = 8
        sizeof(long double)     = 12
~/ieee> fpinfo2
float:
        sizeof(float) =               4
        FLT_MANT_DIG =               24
        machine epsilon =             1.19209e-07  [IEEE 754 32-bit macheps]
        smallest positive number =    1.40130e-45  [IEEE 754 smallest 32-bit subnormal]

double:
        sizeof(double) =              8
        DBL_MANT_DIG =               53
        machine epsilon =             2.22045e-16  [IEEE 754 64-bit macheps]
        smallest positive number =   4.94066e-324  [IEEE 754 smallest 64-bit subnormal]

LONG_DOUBLE:
        sizeof(LONG_DOUBLE) =        12
        LDBL_MANT_DIG =              64
        machine epsilon =             1.08420e-19  [IEEE 754 80-bit macheps]
        smallest positive number =  3.64520e-4951  [not IEEE 754 conformant]

~/ieee> fpshow
Single precision
                       0        0x00000000
                       1        0x3f800000
                      -1        0xbf800000
                       2        0x40000000
                      -2        0xc0000000
             1.19209e-07        0x34000000
            -1.19209e-07        0xb4000000
              1.4013e-45        0x00000001
             -1.4013e-45        0x80000001
             1.17549e-38        0x00800000
            -1.17549e-38        0x80800000
             3.40282e+38        0x7f7fffff
            -3.40282e+38        0xff7fffff
                     inf        0x7f800000
                    -inf        0xff800000
                     nan        0xffc00000

Double precision
                       0        0x00000000 00000000
                       1        0x00000000 3ff00000
                      -1        0x00000000 bff00000
                       2        0x00000000 40000000
                      -2        0x00000000 c0000000
             1.11022e-16        0x00000002 3ca00000
            -1.11022e-16        0x00000002 bca00000
            4.94066e-324        0x00000001 00000000
           -4.94066e-324        0x00000001 80000000
            2.22507e-308        0x00000000 00100000
           -2.22507e-308        0x00000000 80100000
            1.79769e+308        0xffffffff 7fefffff
           -1.79769e+308        0xffffffff ffefffff
                     inf        0x00000000 7ff00000
                    -inf        0x00000000 fff00000
                     nan        0x00000000 fff80000
~/ieee> ofl
Haiku haikubox 1 r28949 Jan 18 2009 18:24:39 BePC Haiku


Begin test: result = inf*zero
Floating-point exception flags = 0x00000001 [ FP_INVALID_OPERATION ]
Inf = inf       Zero = 0        Inf*Zero = nan


Begin test: result = one + inf*zero
Floating-point exception flags = 0x00000001 [ FP_INVALID_OPERATION ]
One = 1 Inf = inf       Zero = 0        One + Inf*Zero = nan


End tests
~/ieee> rwfp2a

~/ieee> rwinfnan2
NaN was written as: nan
Inf was written as: inf
NaN was INCORRECTLY input as 0x00000000 0
Inf was INCORRECTLY input as 0x00000000 0
~/ieee> rwinfnan3
NaN was written as: nan
Inf was written as: inf
NaN was read as: 999 [INCORRECT]
Inf was read as: 999 [INCORRECT]
~/ieee> zerocomp
float       comparision of 0 vs -0: OKAY
double      comparision of 0 vs -0: OKAY
long double comparision of 0 vs -0: OKAY
~/ieee> zerosdq

Zero handling in single-precision IEEE 754 arithmetic:
sizeof(float) = 4
+zero is unsigned               (CORRECT)
-zero is   signed               (CORRECT)
0 - 0 is unsigned               (CORRECT)
(-0) - (+0) is   signed         (CORRECT)
(+1)*(0) is unsigned            (CORRECT)
(-1)*(0) is   signed            (CORRECT)

Zero handling in double-precision IEEE 754 arithmetic:
sizeof(double) = 8
+zero is unsigned               (CORRECT)
-zero is   signed               (CORRECT)
0 - 0 is unsigned               (CORRECT)
(-0) - (+0) is   signed         (CORRECT)
(+1)*(0) is unsigned            (CORRECT)
(-1)*(0) is   signed            (CORRECT)

Zero handling in extended-precision IEEE 754 arithmetic:
sizeof(long double) = 12
+zero is unsigned               (CORRECT)
-zero is unsigned               (WRONG)
0 - 0 is unsigned               (CORRECT)
(-0) - (+0) is unsigned         (WRONG)
(+1)*(0) is unsigned            (CORRECT)
(-1)*(0) is unsigned            (WRONG)
~/ieee> 

[korli]

I believe our GCC2.95 doesn't really support long double type.

[mmlr]

Just to make sure, you're testing this on real hardware? I know from past work with QEMU that some of the FPU code was (and might still be) broken and cause such inaccuracies. It's unlikely that this still is such a big problem, but I thought I'd ask to be sure.

[scottmc]

Replying to mmlr:

Just to make sure, you're testing this on real hardware? I know from past work with QEMU that some of the FPU code was (and might still be) broken and cause such inaccuracies. It's unlikely that this still is such a big problem, but I thought I'd ask to be sure.

yup, on real hardware, an AMDX2 in my case.

[axeld]

This could be a pretty severe problem, and until we've figured out the root cause, I'm bumping this into the alpha1 milestone.

[scottmc]

Artur Wyszynski ran the python regression tests on a gcc4 haiku and posted his test results, see his email to the haikuports dev mailing list here:
​http://lists.ports.haiku-files.org/pipermail/haikuports-devs-ports.haiku-files.org/2009-February/000241.html So it seems that this is not just a gcc2 issue, as most of the same tests still failed. It is curious though that the test_decimal.py passed on the gcc4 haiku.

[oco]

After checking Freepascal about floating point (it seems ok), i have started looking at the difference beetween fpc and gcc.

I have called scanf from fpc passing a pointer to a buffer initialized with 0xFF.

It seems there is at least two problems in the current implementation :

• special values like nan, inf and -inf are not recognized by scanf (or some other internal functions). The buffer remain unchanged in this case. It should explain rwinfnan2 test results : the value 999 remain unchanged after reading some special values.

• when using the long double format, scanf write only the 10 first bytes of the buffer, despite a long double size of 12 bytes. The last 2 bytes remains to 0xFF.

For example, reading "-4.5" with scanf("%Lg", buffer) write this in the buffer :

0x 00 00 00 00 00 00 00 90 01 C0 FF FF

Maybe writing only 10 bytes is OK because it is the maximum size of FPU registers in x86. But is it OK to keep 2 bytes in case the memory was not initialized ? At least, if i am not wrong, the sign should be in the last one, no ?

[bonefish]

Replying to oco:

Maybe writing only 10 bytes is OK because it is the maximum size of FPU registers in x86. But is it OK to keep 2 bytes in case the memory was not initialized ? At least, if i am not wrong, the sign should be in the last one, no ?

No idea. gcc 4.3.1 in Linux also says sizeof(long double) == 12, so I guess that much is correctly at least.

long double support in our gcc 2.95.3 is known to be broken. From what I've heard it is OK in the same gcc version for other platforms, so this is supposedly a gcc configuration problem. I guess looking in gcc's documentation for configuration defines that influence long double behavior and comparing the configurations for other platforms with ours would be a good start to track the problem down.

[oco]

Oh ! I was wrong about the sign. The long double type has a size of 12 bytes but it contains a 10 bytes floating point value (the x87 native format). 2 padding bytes are added to complete the value.

Here is post about this :

• ​http://thread.gmane.org/gmane.linux.c-programming/2780/focus=2782

I will take a look at configuration on other platforms. But i may also try to track down the difference beetween fpc and gcc. This could help me find what kind of configuration i should search for in gcc.

[zooey]

Fixed in hrev31919.