Bug 6777

Summary: lgamma() pole error sets errno to EDOM, should be ERANGE
Product: glibc Reporter: Michael Kerrisk <mtk.manpages>
Component: mathAssignee: Andreas Jaeger <aj>
Status: RESOLVED FIXED    
Severity: normal CC: glibc-bugs
Priority: P2 Flags: fweimer: security-
Version: unspecified   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:
Attachments: test program

Description Michael Kerrisk 2008-07-30 10:55:15 UTC
For the pole error, POSIX.1 says of lgamma():

Pole Error
    The x argument is a negative integer or zero.

    If the integer expression (math_errhandling & MATH_ERRNO) is non-zero, then
errno shall be set to [ERANGE]. If the integer expression (math_errhandling &
MATH_ERREXCEPT) is non-zero, then the divide-by-zero floating-point exception
shall be raised.
==

glibc 2.8's lgamma() does raise the divide-by-zero exception, as expected.

It also sets errno, but to the wrong value: errno is set to EDOM, when it should
instead be ERANGE.
Comment 1 Michael Kerrisk 2008-07-31 08:25:19 UTC
Created attachment 2842 [details]
test program

Attached a test program.  Sample run showing the problem:

$ /tmp/mt_lgamma -- -1
errno == EDOM
fetestexcept() says:  FE_DIVBYZERO
lgamma(-1.00000000000000000e+00)=inf
EDOM FE_DIVBYZERO +inf
Comment 2 Michael Kerrisk 2010-09-11 14:21:40 UTC
This bug appears to have been fixed in glibc 2.10, and man-pages-3.27 is updated
accordingly.