[Bug math/18197] New: atan inaccuracy

["jsm28 at gcc dot gnu.org" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=18197

            Bug ID: 18197
           Summary: atan inaccuracy
           Product: glibc
           Version: 2.21
            Status: NEW
          Severity: normal
          Priority: P2
         Component: math
          Assignee: unassigned at sourceware dot org
          Reporter: jsm28 at gcc dot gnu.org

The dbl-64 version of atan is inaccurate in non-default rounding modes, with
errors exceeding the 9ulp maximum considered acceptable by libm-test.inc.  For
example (on x86_64):

Failure: Test: atan_upward (-0x3.b02d84p-4)
Result:
 is:         -2.26554758731449840114e-01  -0x1.cffbf0fc09b910000000p-3
 should be:  -2.26554758731449507047e-01  -0x1.cffbf0fc09b850000000p-3
 difference:  3.33066907387546962128e-16   0x1.80000000000000000000p-52
 ulp       :  12.0000
 max.ulp   :  1.0000

Failure: Test: atan_towardzero (-0x3.3fb708p-4)
Result:
 is:         -2.00331721927011846551e-01  -0x1.9a478490391a30000000p-3
 should be:  -2.00331721927011541239e-01  -0x1.9a478490391980000000p-3
 difference:  3.05311331771918048616e-16   0x1.60000000000000000000p-52
 ulp       :  11.0000
 max.ulp   :  1.0000

Failure: Test: atan_downward (-0x3.3fb708p-4)
Result:
 is:         -2.00331721927011846552e-01  -0x1.9a478490391a30000000p-3
 should be:  -2.00331721927011568996e-01  -0x1.9a478490391990000000p-3
 difference:  2.77555756156289135105e-16   0x1.40000000000000000000p-52
 ulp       :  10.0000
 max.ulp   :  1.0000

-- 
You are receiving this mail because:
You are on the CC list for the bug.