Bug 13472 - PPC64 hypot() is "inf" for very large values that are non-infinite on other architectures
Summary: PPC64 hypot() is "inf" for very large values that are non-infinite on other a...
Alias: None
Product: glibc
Classification: Unclassified
Component: math (show other bugs)
Version: 2.14
: P2 normal
Target Milestone: ---
Assignee: Andreas Jaeger
Depends on:
Reported: 2011-12-05 23:36 UTC by Dave Malcolm
Modified: 2014-06-27 11:32 UTC (History)
1 user (show)

See Also:
Last reconfirmed:
fweimer: security-


Note You need to log in before you can comment on or make changes to this bug.
Description Dave Malcolm 2011-12-05 23:36:53 UTC
ppc64's glibc appears to have an optimized implementation of "hypot" (in libm):

This seems to have different behavior near infinity when compared to other architectures.

This leads to CPython's test suite failing on ppc64 with glibc-2.14.90-19.ppc64 (albeit with Fedora's build of glibc) within the "cmath" module (math on complex numbers):



CPython's "cmath" module internally uses hypot() in various cases of very large numbers in order to avoid overflow.  All of the failing test cases turned out to involve (at the C layer) a call to hypot() with a pair of very large finite values, where the result is very large but finite on other architectures, but is "inf" on ppc64, leading to various unexpected values, and an architecture-specific failure of this test.

"man hypot" says:
> The calculation is performed without undue overflow or underflow during
> the intermediate steps of the calculation.

Test case:
Python's math.hypot() is a thin wrapper around hypot(3); a Python "float" uses a C "double" internally.

https://bugzilla.redhat.com/attachment.cgi?id=541097 is a Python script to exercise hypot(3).  It calls hypot() for all of the x,y pairs that fail within the Python test case on ppc64.

On x86_64, with glibc-2.14-5.x86_64 (and python-2.7.1-7.fc15.x86_64), all results are non-infinite, with exponent e+307 or e+308 (see:
for the results on this box).

On ppc64, with glibc-2.14.90-19.ppc64 (and python-2.7.2-4.2.fc16.ppc64), all results are "inf".

(hopefully the attachments on that other bugzilla instance are readable).
Comment 1 Andreas Schwab 2011-12-06 10:45:50 UTC
Fixed on master.
Comment 2 Jakub Jelinek 2011-12-06 13:19:52 UTC
Can you explain where and how?  I don't see any recent fixes for the buggy sysdeps/powerpc/fpu/e_hypot.c, all other architectures use a sane implementation, but this new powerpc implementation doesn't attempt to handle any corner cases except for the checks if larger one is > two500 or smaller one < twoM500.
Comment 3 Andreas Schwab 2011-12-06 13:26:56 UTC
Forgot to push out.
Comment 4 Dave Malcolm 2011-12-06 15:11:04 UTC
Note to self:
  Fix appears to be: