hypot() raises an exception for underflow and overflow, but only sets errno to ERANGE for the overflow case. It should set errno also for the underflow case. Background: On error, many glibc math functions both set errno and raise an exception (fetestexcept(3)). For example, the following function all do this: acos(), asin(), cosh(), sinh(), acosh(), asinh(), exp(), exp2(), ldexp(), log(), log10(), log2(). However, there is much inconsistency. Some functions raise an exception, but don't set errno. Some functions set errno for some errors, but not others. A few set errno, but don't raise an exception. This series of bug reports documents deviations from what I consider the ideal: all functions should BOTH set errno AND raise an exception for all errors. All of these reports relate to tests on glibc 2.8 (as provided by SUSE 11.0).
Created attachment 2857 [details] test program Sample runs demonstrating the problem: $ /tmp/mt_hypot dbl_max dbl_max errno == ERANGE fetestexcept() says: FE_OVERFLOW FE_INEXACT hypot(1.79769313486231571e+308,1.79769313486231571e+308)=inf ERANGE FE_OVERFLOW +inf $ /tmp/mt_hypot subnormal:20 subnormal:20 errno == 0 fetestexcept() says: FE_UNDERFLOW FE_INEXACT hypot(2.12199579096527232e-314,2.12199579096527232e-314)=3.00095522690537735e-314 0 FE_UNDERFLOW subnormal
Confirmed with current sources on both x86 and x86_64.
It's now documented that errno is only intended to be set to ERANGE for underflow to 0, not for underflow to subnormals, and hypot can't underflow to 0.