Bug 13472

Summary: PPC64 hypot() is "inf" for very large values that are non-infinite on other architectures
Product: glibc Reporter: Dave Malcolm <dmalcolm>
Component: mathAssignee: Andreas Jaeger <aj>
Status: RESOLVED FIXED    
Severity: normal CC: jakub
Priority: P2 Flags: fweimer: security-
Version: 2.14   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:

Description Dave Malcolm 2011-12-05 23:36:53 UTC
ppc64's glibc appears to have an optimized implementation of "hypot" (in libm):
  sysdeps/powerpc/fpu/e_hypot.c

This seems to have different behavior near infinity when compared to other architectures.

This leads to CPython's test suite failing on ppc64 with glibc-2.14.90-19.ppc64 (albeit with Fedora's build of glibc) within the "cmath" module (math on complex numbers):

  https://bugzilla.redhat.com/show_bug.cgi?id=750811

  http://bugs.python.org/issue13534

CPython's "cmath" module internally uses hypot() in various cases of very large numbers in order to avoid overflow.  All of the failing test cases turned out to involve (at the C layer) a call to hypot() with a pair of very large finite values, where the result is very large but finite on other architectures, but is "inf" on ppc64, leading to various unexpected values, and an architecture-specific failure of this test.

"man hypot" says:
> The calculation is performed without undue overflow or underflow during
> the intermediate steps of the calculation.

Test case:
Python's math.hypot() is a thin wrapper around hypot(3); a Python "float" uses a C "double" internally.

https://bugzilla.redhat.com/attachment.cgi?id=541097 is a Python script to exercise hypot(3).  It calls hypot() for all of the x,y pairs that fail within the Python test case on ppc64.

On x86_64, with glibc-2.14-5.x86_64 (and python-2.7.1-7.fc15.x86_64), all results are non-infinite, with exponent e+307 or e+308 (see:
  https://bugzilla.redhat.com/attachment.cgi?id=541102
for the results on this box).

On ppc64, with glibc-2.14.90-19.ppc64 (and python-2.7.2-4.2.fc16.ppc64), all results are "inf".

(hopefully the attachments on that other bugzilla instance are readable).
Comment 1 Andreas Schwab 2011-12-06 10:45:50 UTC
Fixed on master.
Comment 2 Jakub Jelinek 2011-12-06 13:19:52 UTC
Can you explain where and how?  I don't see any recent fixes for the buggy sysdeps/powerpc/fpu/e_hypot.c, all other architectures use a sane implementation, but this new powerpc implementation doesn't attempt to handle any corner cases except for the checks if larger one is > two500 or smaller one < twoM500.
Comment 3 Andreas Schwab 2011-12-06 13:26:56 UTC
Forgot to push out.
Comment 4 Dave Malcolm 2011-12-06 15:11:04 UTC
Note to self:
  Fix appears to be:
    http://repo.or.cz/w/glibc.git/commit/850fb039cec802072f70ed9763927881bbbf639c