known maximal errors
Paul Zimmermann
Paul.Zimmermann@inria.fr
Thu Feb 9 10:02:58 GMT 2023
Hi,
[1] gives a maximal known error of 0 ulp for hypotf on x86_64, but the
following gives an error of 1 ulp:
Checking hypot with glibc-2.37
hypot 0 -1 -0x1.003222p-20,-0x1.6a2d58p-32 [0.501] 0.500001 0.500000001392678
libm gives 0x1.003224p-20
mpfr gives 0x1.003222p-20
Same for binary64 tan:
tan 0 -1 -0x1.317cd745dd37cp+9 [1] [0.619] 0.618367 0.6183663571802361
libm gives -0x1.efcf403abf99ep+3
mpfr gives -0x1.efcf403abf99fp+3
and for atan2:
atan2 0 -1 0x1.ed6060626eecfp-429,0x1.f42ebb62994dcp-426 [0.524] 0.523171 0.5231707007273592
libm gives 0x1.f67f36f292b58p-4
mpfr gives 0x1.f67f36f292b59p-4
For binary64 sin a maximal known error of 2 ulps is given but I can only
find an error of 0.516 ulps, thus rounding to 1.
Do you have an example with 2 ulps on x86_64 ?
Best regards,
Paul
[1] https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html
More information about the Libc-alpha
mailing list