Bug 5350 - floor() is giving incorrect results on alpha
Summary: floor() is giving incorrect results on alpha
Alias: None
Product: glibc
Classification: Unclassified
Component: math (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: rth@redhat.com
Depends on:
Reported: 2007-11-16 22:01 UTC by Aurelien Jarno
Modified: 2014-07-03 12:32 UTC (History)
1 user (show)

See Also:
Host: alphaev68-unknown-linux-gnu
Target: alphaev68-unknown-linux-gnu
Build: alphaev68-unknown-linux-gnu
Last reconfirmed:
fweimer: security-

Patch to revert the code to the old version (1.28 KB, patch)
2007-11-16 22:02 UTC, Aurelien Jarno
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aurelien Jarno 2007-11-16 22:01:30 UTC
floor() is giving wrong result for some corner cases on alpha, as shown by the 
following testcase:

#include <assert.h>
#include <float.h>
#include <math.h>
#include <stdio.h>

int main() {
  double x;

  printf("%s%d\n", "DBL_MANT_DIG = ", DBL_MANT_DIG);
  x = ldexp (1.0, DBL_MANT_DIG) - 1.0;
  printf("%lf %lf\n", x, floor(x));
  assert(x == floor(x));    /* does not work in alpha */
  return 0;

It is actually a regression introduced by the following changesets:

2007-03-14  Richard Henderson  <rth@redhat.com>

        * sysdeps/alpha/fpu/s_ceil.c: Rewrite without branches.
        * sysdeps/alpha/fpu/s_ceilf.c: Likewise.
        * sysdeps/alpha/fpu/s_floor.c: Likewise.
        * sysdeps/alpha/fpu/s_floorf.c: Likewise.
        * sysdeps/alpha/fpu/s_rint.c: Likewise.
        * sysdeps/alpha/fpu/s_rintf.c: Likewise.

This change also assumes that the internal FPU representation has the same 
precision as the IEEE one, which is not always true.
Comment 1 Aurelien Jarno 2007-11-16 22:02:56 UTC
Created attachment 2094 [details]
Patch to revert the code to the old version
Comment 2 Richard Henderson 2010-05-04 16:23:20 UTC
Patch applied: 116ff9ad1801108d084fd9ff94241f26a33dff1c