incorrectly rounded square root

Paul Zimmermann Paul.Zimmermann@inria.fr
Wed Jun 2 07:51:17 GMT 2021


       Hi Jeff,

thank you for your answer. After investigating more it seems fesetround()
is ineffective with newlib. Indeed if I print the result of fegetround()
just after the fesetround() calls I get:

fegetround: 0
RNDN: 0x1.ff83fp+63
fegetround: 0
RNDZ: 0x1.ff83fp+63
fegetround: 0
RNDU: 0x1.ff83fp+63
fegetround: 0
RNDD: 0x1.ff83fp+63

whereas with GNU libc I get:

fegetround: 0
RNDN: 0x1.ff83fp+63
fegetround: 3072
RNDZ: 0x1.ff83eep+63
fegetround: 2048
RNDU: 0x1.ff83fp+63
fegetround: 1024
RNDD: 0x1.ff83eep+63

Thus it seems this has nothing to do with the square root.

According to gdb the fesetround() code used is the following:

(gdb) break fesetround
Breakpoint 1 at 0x1650: file ../../../../../../newlib/libm/machine/x86_64/fenv.c, line 371.

Paul




More information about the Newlib mailing list