The ldbl-128ibm implementation of modfl has several issues: * In the case for high part exponent < 52, (((i0&i)|(i1&0x7fffffffffffffffLL))==0) is not a correct test for integer argument; the latter part effectively tests just if the low part is a power of 2. Example: an argument of 0x1.ffffffffffffffp51L is wrongly treated as being an integer. * In that case, the handling of non-integers when detected as such fails to allow for the low part being of opposite sign to the integer high part. Example: 0x1.ffffffffffffff1p51L produces integer part 0x1p+52, fractional part -0x1.ep-6. * The case of high part exponent > 103 is treated as meaning the argument is an integer, but that's not true even when restricted to numbers with 106 contiguous mantissa bits. Example: 0x1.ffffffffffffffffffffffffff8p104L is wrongly treated as being an integer. * In the case for exponents 52 to 103, the test i1&i)==0 for being an integer is completely bogus; the mask i = -1ULL>>(j0-52) is computed as if this were a conventional IEEE format. Example: 0x0.fffffffffffffcp52L is wrongly treated as being an integer. * When something is detected as not an integer in that case, the INSERT_WORDS64 (xlo, i1&(~i)) is always inserting a subnormal / zero value in the low part, its exponent having been masked off earlier. Example: 0x1.00000000000001111p56L gets integer part 0x1p+56 and fractional part 0x1.111p+0, which is more than 1.
This has been filed today as a gcc bug when it is a glibc bug, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96600 for details.