[PATCH] fix for accuracy problem in atan2f

[Bob Wilson]

There is a problem with the version of atan2f in 
sysdeps/ieee754/flt-32/e_atan2f.c that appears to be a typo in the value 
of pi_lo, and that causes the results to be slightly inaccurate for some 
input values.  The attached test program will demonstrate the problem on 
a platform that uses this version of atan2f.  (I don't see the problem 
on i386, presumably because it's using a machine-specific version of 
atan2f, but I do see it on x86_64 with RedHat Enterprise 3.)

Back in July 1997, the value of pi in this file (back when it was in 
sysdeps/libm-ieee754/) was changed to be the correct round-to-nearest 
value, and pi_lo was adjusted accordingly.  It appears that the exponent 
of pi_lo was mistakenly left unchanged from its previous value.  There 
are several indications of this:

-  The hex value of pi_lo shown in comments is 0xb3bbbd2e, which 
corresponds to -8.7422776573e-08, but the float value actually used has 
the same mantissa but an exponent of "e-07".

- The values of pi and pi_lo should add up to the real value of pi (as 
much as can be represented).  The exponent should be "e-08" for this to 
be true.

- The test case below gets the wrong answer (atan2f != (float)atan2) 
when pi_lo has an exponent of "e-07" but gets the correct answer when 
the exponent is changed to "e-08".

The testcase and patch are attached below.  Here is a ChangeLog entry:

2005-07-20  Bob Wilson  <bob.wilson@acm.org>
            Darin Petkov <darin@tensilica.com>

        * sysdeps/ieee754/flt-32/e_atan2f.c (pi_lo): Correct exponent value.


 

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: atan2f-test.c
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20050720/9be7d1f6/attachment.c>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: atan2f.patch
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20050720/9be7d1f6/attachment.ksh>