incorrectly rounded square root
Paul Zimmermann
Paul.Zimmermann@inria.fr
Wed Jun 2 07:51:17 GMT 2021
Hi Jeff,
thank you for your answer. After investigating more it seems fesetround()
is ineffective with newlib. Indeed if I print the result of fegetround()
just after the fesetround() calls I get:
fegetround: 0
RNDN: 0x1.ff83fp+63
fegetround: 0
RNDZ: 0x1.ff83fp+63
fegetround: 0
RNDU: 0x1.ff83fp+63
fegetround: 0
RNDD: 0x1.ff83fp+63
whereas with GNU libc I get:
fegetround: 0
RNDN: 0x1.ff83fp+63
fegetround: 3072
RNDZ: 0x1.ff83eep+63
fegetround: 2048
RNDU: 0x1.ff83fp+63
fegetround: 1024
RNDD: 0x1.ff83eep+63
Thus it seems this has nothing to do with the square root.
According to gdb the fesetround() code used is the following:
(gdb) break fesetround
Breakpoint 1 at 0x1650: file ../../../../../../newlib/libm/machine/x86_64/fenv.c, line 371.
Paul
More information about the Newlib
mailing list