Bug 4407 - signgam value for lgamma on special values
Summary: signgam value for lgamma on special values
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: math (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Andreas Jaeger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-04-22 22:39 UTC by Vincent Lefèvre
Modified: 2014-07-04 22:15 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vincent Lefèvre 2007-04-22 22:39:01 UTC
The signgam value is incorrect for lgamma(-0.), at least on x86 and PowerPC (and
various glibc versions): it is 1, whereas it should be -1 (since tgamma(-0.) is
-inf). For instance, sysdeps/ieee754/dbl-64/e_lgamma_r.c contains:

        if((ix|lx)==0) return one/fabs(x);

i.e. it lets signgam to the value 1, even when x is -0. Other files contains the
same error.

Moreover, on values where tgamma is NaN, the glibc sets signgam to 1. But is
this correct? The POSIX standard is silent on this point, but as the sign isn't
defined, the value of signgam could be left unchanged (and I don't think that
taking into account the sign of the NaN returned by tgamma doesn't really make
much sense). In any case, this should be documented.
Comment 1 Jakub Jelinek 2007-04-23 14:19:27 UTC
For lgamma (-0.) according to
http://www.opengroup.org/onlinepubs/009695399/functions/lgamma.html
If x is a non-positive integer, a pole error should occur and +HUGE_VAL be
returned, the same as for lgamma (0.).
That's different from tgamma
http://www.opengroup.org/onlinepubs/009695399/functions/tgamma.html
which has:
If x is +-0, a pole error shall occur, and tgamma(), tgammaf(), and tgammal()
shall return +-HUGE_VAL, +-HUGE_VALF, and +-HUGE_VALL, respectively.

As for the second part, I believe the current behavior is ok, NaNs don't have
a sign and thus the content of signgam after lgamma(NAN) is IMNSHO undefined.
Comment 2 Vincent Lefèvre 2007-04-23 15:54:48 UTC
(In reply to comment #1)
> For lgamma (-0.) according to
> http://www.opengroup.org/onlinepubs/009695399/functions/lgamma.html
> If x is a non-positive integer, a pole error should occur and +HUGE_VAL be
> returned, the same as for lgamma (0.).

Yes, the return value is correct. But the signgam value (which is the sign of
the Gamma function, i.e. tgamma) is not for lgamma(-0.).
Comment 3 Jakub Jelinek 2007-04-27 10:07:32 UTC
FYI, Solaris for:
#define _XOPEN_SOURCE 600
#include <math.h>
#include <stdio.h>

double zero = 0.0;
double mzero;

int
main (void)
{
  double d;
  mzero = copysign (zero, -1.0);
  d = lgamma (zero);
  printf ("%g %d\n", d, signgam);
  d = lgamma (mzero);
  printf ("%g %d\n", d, signgam);
  return 0;
}
also prints Inf 1 twice.
Comment 4 Vincent Lefèvre 2007-04-27 12:39:16 UTC
But both HP-UX and OSF1 return the correct sign -1 (though OSF1 has other bugs).
Comment 5 Ulrich Drepper 2007-10-06 18:37:53 UTC
I've changed the code to store -1.