[PATCH] PowerPC - Fix PPC64 floorl

[Joseph S. Myers]

On Thu, 19 Apr 2012, Steven Munroe wrote:

> > I don't see how that can be correct.  If the high part is small, but an 
> > integer, the result of this function should still depend on whether the 
> > low part is positive, negative or zero.  Presumably the testsuite failed 
> > to detect that bug in this patch, meaning you also need tests (conditioned 
> > on TEST_LDOUBLE - they can be written to be valid for all long double 
> > formats) for values that when rounded to 53 bits are an integer, but 
> > differ from the integer value in lower bits (covering at least the four 
> > cases of positive and negative value, above and below the exact integer it 
> > is close to).
> 
> Are you concerned that xh may be "prerounded" away (from default
> rounding mode)? This is not what we want for floor. In which case we
> should "deround" xh before the __floor operation.
> Perhaps:
> 
> +  if (__builtin_expect(__builtin_fabs (x) < two52, 1))
> +      return ldbl_pack (__floor (xh+xl), -0.0);

That won't help either.

In a valid long double, xh is always the value of the long double, rounded 
to nearest, independent of the current rounding direction; that's the 
definition of the format.  The result of floorl doesn't depend on the 
current rounding direction either; it always rounds to an integer value in 
the direction of negative infinity.

If xh is an integer, and xl is positive or zero with absolute value less 
than 1, xh is the correct result of floorl.  If xh is an integer, and xl 
is negative with absolute value less than 1, the correct result of floorl 
is xh - 1.  (If xl has absolute value at least 1, which means xh is large, 
then the correct result is xh + floor(xl); that formula is indeed correct 
whenever xh is an integer.  And if xh is not an integer, the correct 
result is floor (xh).)

The sort of case where I expect this patch will get things wrong is floorl 
(0x1p51L - 0x1p-10L) or floorl (-0x1p51L - 0x1p-10L).

-- 
Joseph S. Myers
joseph@codesourcery.com