known maximal errors

Paul Zimmermann Paul.Zimmermann@inria.fr
Thu Feb 9 10:02:58 GMT 2023


       Hi,

[1] gives a maximal known error of 0 ulp for hypotf on x86_64, but the
following gives an error of 1 ulp:

Checking hypot with glibc-2.37
hypot 0 -1 -0x1.003222p-20,-0x1.6a2d58p-32 [0.501] 0.500001 0.500000001392678
libm gives 0x1.003224p-20
mpfr gives 0x1.003222p-20

Same for binary64 tan:

tan 0 -1 -0x1.317cd745dd37cp+9 [1] [0.619] 0.618367 0.6183663571802361
libm gives -0x1.efcf403abf99ep+3
mpfr gives -0x1.efcf403abf99fp+3

and for atan2:

atan2 0 -1 0x1.ed6060626eecfp-429,0x1.f42ebb62994dcp-426 [0.524] 0.523171 0.5231707007273592
libm gives 0x1.f67f36f292b58p-4
mpfr gives 0x1.f67f36f292b59p-4

For binary64 sin a maximal known error of 2 ulps is given but I can only
find an error of 0.516 ulps, thus rounding to 1.
Do you have an example with 2 ulps on x86_64 ?

Best regards,
Paul

[1] https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html


More information about the Libc-alpha mailing list