[PATCH] math: do not use __builtin_fpclassify on alpha

[Matt Turner]

__builtin_fpclassify expands on alpha into FP compares (cmpteq, cmptle,
cmptun) without the software-completion suffix. What happens to a
subnormal operand then depends on the kernel's denormal-operand trap
setting (FPCR_DNOD): with denormal traps disabled, hardware silently
flushes the operand to zero and the builtin reports FP_ZERO instead of
FP_SUBNORMAL. With denormal traps enabled, the compare traps into the
kernel, but carries no software-completion bit for the kernel to emulate
it, so the kernel delivers SIGFPE unconditionally.

  $ cat fpc.c
  #include <math.h>
  #include <stdio.h>
  int main (void)
  {
    volatile double sub = 0x1p-1050;
    printf ("__fpclassify: %d\n", __fpclassify (sub));
    fflush (stdout);
    printf ("builtin     : %d\n", fpclassify (sub));
    return 0;
  }
  $ ./fpc
  __fpclassify: 3
  Floating point exception (core dumped)

Confirmed on an Alpha EV68 running a kernel with denormal traps enabled
by default: classifying a subnormal double kills the process.

__fpclassify uses integer bit extraction, so it is unaffected.  Use it
on alpha instead of the builtin.
---
 math/math.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git ./math/math.h ./math/math.h
index 75830d7696..32ca994cb7 100644
--- ./math/math.h
+++ ./math/math.h
@@ -1091,9 +1091,16 @@ enum
    so disable builtins if this is enabled.  When fixed in a newer GCC,
    the __SUPPORT_SNAN__ check may be skipped for those versions.  */
 
+/* On Alpha, GCC expands __builtin_fpclassify into FP compares (cmpteq,
+   cmptle, cmptun) without the software-completion suffix.  A subnormal
+   operand then either misclassifies (denormal traps disabled, flushed to
+   zero) or crashes (denormal traps enabled, no completion path for the
+   trap).  Use __fpclassify (integer bit-extraction) instead.  */
+
 /* Return number of classification appropriate for X.  */
 # if ((__GNUC_PREREQ (4,4) && !defined __SUPPORT_SNAN__)		      \
       || __glibc_clang_prereq (2,8))					      \
+     && !defined __alpha__						      \
      && (!defined __OPTIMIZE_SIZE__ || defined __cplusplus)
      /* The check for __cplusplus allows the use of the builtin, even
 	when optimization for size is on.  This is provided for
-- 
2.54.0