Bug 13472 - PPC64 hypot() is "inf" for very large values that are non-infinite on other architectures
Summary: PPC64 hypot() is "inf" for very large values that are non-infinite on other a...
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: math (show other bugs)
Version: 2.14
: P2 normal
Target Milestone: ---
Assignee: Andreas Jaeger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-12-05 23:36 UTC by Dave Malcolm
Modified: 2014-06-27 11:32 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dave Malcolm 2011-12-05 23:36:53 UTC
ppc64's glibc appears to have an optimized implementation of "hypot" (in libm):
  sysdeps/powerpc/fpu/e_hypot.c

This seems to have different behavior near infinity when compared to other architectures.

This leads to CPython's test suite failing on ppc64 with glibc-2.14.90-19.ppc64 (albeit with Fedora's build of glibc) within the "cmath" module (math on complex numbers):

  https://bugzilla.redhat.com/show_bug.cgi?id=750811

  http://bugs.python.org/issue13534

CPython's "cmath" module internally uses hypot() in various cases of very large numbers in order to avoid overflow.  All of the failing test cases turned out to involve (at the C layer) a call to hypot() with a pair of very large finite values, where the result is very large but finite on other architectures, but is "inf" on ppc64, leading to various unexpected values, and an architecture-specific failure of this test.

"man hypot" says:
> The calculation is performed without undue overflow or underflow during
> the intermediate steps of the calculation.

Test case:
Python's math.hypot() is a thin wrapper around hypot(3); a Python "float" uses a C "double" internally.

https://bugzilla.redhat.com/attachment.cgi?id=541097 is a Python script to exercise hypot(3).  It calls hypot() for all of the x,y pairs that fail within the Python test case on ppc64.

On x86_64, with glibc-2.14-5.x86_64 (and python-2.7.1-7.fc15.x86_64), all results are non-infinite, with exponent e+307 or e+308 (see:
  https://bugzilla.redhat.com/attachment.cgi?id=541102
for the results on this box).

On ppc64, with glibc-2.14.90-19.ppc64 (and python-2.7.2-4.2.fc16.ppc64), all results are "inf".

(hopefully the attachments on that other bugzilla instance are readable).
Comment 1 Andreas Schwab 2011-12-06 10:45:50 UTC
Fixed on master.
Comment 2 Jakub Jelinek 2011-12-06 13:19:52 UTC
Can you explain where and how?  I don't see any recent fixes for the buggy sysdeps/powerpc/fpu/e_hypot.c, all other architectures use a sane implementation, but this new powerpc implementation doesn't attempt to handle any corner cases except for the checks if larger one is > two500 or smaller one < twoM500.
Comment 3 Andreas Schwab 2011-12-06 13:26:56 UTC
Forgot to push out.
Comment 4 Dave Malcolm 2011-12-06 15:11:04 UTC
Note to self:
  Fix appears to be:
    http://repo.or.cz/w/glibc.git/commit/850fb039cec802072f70ed9763927881bbbf639c