New GNU C Library (glibc) security flaw reported on 30 Jan 2024

[Xi Ruoyao]

On Wed, 2024-01-31 at 12:52 -0300, Adhemerval Zanella Netto wrote:

/* snip */

> 
> I see this is an manual issue rather than a GNU 'extension' to qsort semantic.
> And I think we should fix BZ#31322 by using a transitive comparison instead of
> trying to support such cases.

To me the documentation is correct (though arguably in a very subtle
way):

   Here is an example of a comparison function which works with an array
of numbers of type ‘double’:

     int
     compare_doubles (const void *a, const void *b)
     {
       const double *da = (const double *) a;
       const double *db = (const double *) b;

       return (*da > *db) - (*da < *db);
     }

It says "numbers."  But NaN literally means, "Not a Number."  C23 says:

Floating types shall be able to represent signed zeros or an unsigned
zero and all normalized floating-point numbers. In addition, floating
types may be able to contain other kinds of floating-point numbers, such
as subnormal floating-point numbers and unnormalized floating-point
numbers, and values that are not floating-point numbers, such as NaNs
and (signed and unsigned) infinities.  A NaN is a value signifying Not-
a-Number.

So at least in C23 it's clear that NaN and infinite values are not
numbers.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University