Bug 6795 - hypot() does not set errno consistently on error
Summary: hypot() does not set errno consistently on error
Alias: None
Product: glibc
Classification: Unclassified
Component: math (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
Depends on:
Reported: 2008-07-31 09:54 UTC by Michael Kerrisk
Modified: 2014-07-04 05:44 UTC (History)
1 user (show)

See Also:
Last reconfirmed:
fweimer: security-

test program (2.74 KB, text/plain)
2008-07-31 09:56 UTC, Michael Kerrisk

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Kerrisk 2008-07-31 09:54:40 UTC
hypot() raises an exception for underflow and overflow, but only sets errno to
ERANGE for the overflow case.  It should set errno also for the underflow case.

On error, many glibc math functions both set errno and raise an exception
(fetestexcept(3)).  For example, the following  function all do this: acos(),
asin(), cosh(), sinh(), acosh(), asinh(), exp(), exp2(), ldexp(), log(),
log10(), log2().  However, there is much inconsistency.  Some functions raise an
exception, but don't set errno.  Some functions set errno for some errors, but
not others.  A few set errno, but don't raise an exception.  This series of bug
reports documents deviations from what I consider the ideal: all functions
should BOTH set errno AND raise an exception for all errors.

All of these reports relate to tests on glibc 2.8 (as provided by SUSE 11.0).
Comment 1 Michael Kerrisk 2008-07-31 09:56:27 UTC
Created attachment 2857 [details]
test program

Sample runs demonstrating the problem:

$ /tmp/mt_hypot dbl_max dbl_max
errno == ERANGE
fetestexcept() says:  FE_OVERFLOW FE_INEXACT

$ /tmp/mt_hypot subnormal:20 subnormal:20
errno == 0
fetestexcept() says:  FE_UNDERFLOW FE_INEXACT

0 FE_UNDERFLOW subnormal
Comment 2 Joseph Myers 2012-03-01 02:28:33 UTC
Confirmed with current sources on both x86 and x86_64.
Comment 3 Joseph Myers 2013-11-28 16:58:14 UTC
It's now documented that errno is only intended to be set to ERANGE for underflow to 0, not for underflow to subnormals, and hypot can't underflow to 0.