Bug 18031

Summary: [ldbl-128ibm] modfl problems
Product: glibc Reporter: Joseph Myers <jsm28>
Component: mathAssignee: Not yet assigned to anyone <unassigned>
Status: NEW ---    
Severity: normal CC: fw, jakub
Priority: P2 Flags: fweimer: security-
Version: 2.21   
Target Milestone: ---   
URL: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96600
See Also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96600
Host: Target:
Build: Last reconfirmed:

Description Joseph Myers 2015-02-25 23:07:39 UTC
The ldbl-128ibm implementation of modfl has several issues:

* In the case for high part exponent < 52, (((i0&i)|(i1&0x7fffffffffffffffLL))==0) is not a correct test for integer argument; the latter part effectively tests just if the low part is a power of 2.  Example: an argument of 0x1.ffffffffffffffp51L is wrongly treated as being an integer.

* In that case, the handling of non-integers when detected as such fails to allow for the low part being of opposite sign to the integer high part.  Example: 0x1.ffffffffffffff1p51L produces integer part 0x1p+52, fractional part -0x1.ep-6.

* The case of high part exponent > 103 is treated as meaning the argument is an integer, but that's not true even when restricted to numbers with 106 contiguous mantissa bits.  Example: 0x1.ffffffffffffffffffffffffff8p104L is wrongly treated as being an integer.

* In the case for exponents 52 to 103, the test i1&i)==0 for being an integer is completely bogus; the mask i = -1ULL>>(j0-52) is computed as if this were a conventional IEEE format.  Example: 0x0.fffffffffffffcp52L is wrongly treated as being an integer.

* When something is detected as not an integer in that case, the INSERT_WORDS64 (xlo, i1&(~i)) is always inserting a subnormal / zero value in the low part, its exponent having been masked off earlier.  Example: 0x1.00000000000001111p56L gets integer part 0x1p+56 and fractional part 0x1.111p+0, which is more than 1.
Comment 1 Jakub Jelinek 2020-08-13 11:37:12 UTC
This has been filed today as a gcc bug when it is a glibc bug, see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96600 for details.