Serious __fpclassify bug

[Joseph E. Vornehm, Jr.]

There is an apparent typo with serious consequences in 
newlib/libm/common/s_fpclassify.c.  I've attached a patch for newlib 
1.11.0.  (cvsweb seems to indicate there haven't been any changes to 
that file recently.)

I first noticed the problem when fmax(-12.0, 0.0) returned -12.0.  I 
looked through the code, and since the values in question in 
__fpclassify{f,d} are unsigned ints, some of the comparisons are always 
false (e.g., (w >= a && w <= b) where a > b).  As a result, negative 
numbers were being classified as FP_NAN and thence being returned 
without comparison by fmax().  I tried the patched version, and now 
fmax(-12.0, 0.0) returns 0.0 like it should.

I was only concerned about fmax() and fmin(); I'm not sure what else is 
affected by this, though I would guess it's a fair bit.

Joe Vornehm
The MITRE Corporation


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: fpclassify-patch.txt
URL: <http://sourceware.org/pipermail/newlib/attachments/20030627/84e85ed3/attachment.txt>