Bug 6809

Summary: tgamma() sets errno inconsistently for domain error
Product: glibc Reporter: Michael Kerrisk <mtk.manpages>
Component: mathAssignee: Not yet assigned to anyone <unassigned>
Status: RESOLVED FIXED    
Severity: normal CC: glibc-bugs
Priority: P2 Flags: fweimer: security-
Version: unspecified   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:
Attachments: test program

Description Michael Kerrisk 2008-07-31 13:34:14 UTC
The POSIX.1 specification of tgamma says:

==
The value of x is a negative integer, or x is -Inf.

If the integer expression (math_errhandling & MATH_ERRNO) is non-zero, then
errno shall be set to [EDOM]. If the integer expression (math_errhandling &
MATH_ERREXCEPT) is non-zero, then the invalid floating-point exception shall be
raised.

==

For both -ve integer and -Inf, GNU's tgamma() correctly raises an FE_INVALID
exception.  However, errno is only correctly set (to EDOM) for the -ve integer
case.  For -Inf, errno is not set; it shoul dbe set to EDOM.

Background: 
On error, many glibc math functions both set errno and raise an exception
(fetestexcept(3)).  For example, the following  function all do this: acos(),
asin(), cosh(), sinh(), acosh(), asinh(), exp(), exp2(), ldexp(), log(),
log10(), log2().  However, there is much inconsistency.  Some functions raise an
exception, but don't set errno.  Some functions set errno for some errors, but
not others.  A few set errno, but don't raise an exception.  This series of bug
reports documents deviations from what I consider the ideal: all functions
should BOTH set errno AND raise an exception for all errors.

All of these reports relate to tests on glibc 2.8 (as provided by SUSE 11.0).
Comment 1 Michael Kerrisk 2008-07-31 13:35:33 UTC
Created attachment 2881 [details]
test program

Test runs demonstrating the problem:

$ /tmp/mt_tgamma -- -1
errno == EDOM
fetestexcept() says:  FE_INVALID
tgamma(-1.00000000000000000e+00)=nan
EDOM FE_INVALID nan

$ /tmp/mt_tgamma -- -inf
errno == 0
fetestexcept() says:  FE_INVALID
tgamma(-inf)=nan
0 FE_INVALID nan
Comment 2 Joseph Myers 2012-03-01 23:27:08 UTC
Confirmed with current sources on both x86 and x86_64.
Comment 3 Joseph Myers 2013-05-10 11:35:59 UTC
Fixed for 2.18 by:

commit ab2b94613d85ba9edc659bd5e930b9ab1ec8e8a9
Author: Joseph Myers <joseph@codesourcery.com>
Date:   Fri May 10 11:35:11 2013 +0000

    Fix tgamma errno setting on domain error (bug 6809).