New GNU C Library (glibc) security flaw reported on 30 Jan 2024

[Adhemerval Zanella Netto]

On 31/01/24 11:55, Vincent Lefevre wrote:
> On 2024-01-31 22:23:32 +0800, Xi Ruoyao wrote:
>> On Wed, 2024-01-31 at 14:08 +0000, Turritopsis Dohrnii Teo En Ming
>> wrote:
>>> Subject: New GNU C Library (glibc) security flaw reported on 30 Jan 2024
>>>
>>> Good day from Singapore,
>>>
>>> I recently stumbled upon this insightful article and wanted to share it with you.
>>>
>>> Article: New Linux glibc flaw lets attackers get root on major distros
>>> Link: https://www.bleepingcomputer.com/news/security/new-linux-glibc-flaw-lets-attackers-get-root-on-major-distros/
>>
>> I cannot see why https://www.qualys.com/2024/01/30/qsort.txt is a
>> **Glibc** security issue.  The standard is clear that if you pass a non-
>> transitive comparator to qsort, you invoke an undefined behavior.
> 
> This is what the ISO C standard says. But the glibc manual explicitly
> allows non-transitive comparators.
> 
> See the example in 9.1 Defining the Comparison Function:
> 
>    Here is an example of a comparison function which works with an array
> of numbers of type ‘double’:
> 
>      int
>      compare_doubles (const void *a, const void *b)
>      {
>        const double *da = (const double *) a;
>        const double *db = (const double *) b;
> 
>        return (*da > *db) - (*da < *db);
>      }
> 
> The non-transitivity can be demonstrated with the following test
> program:
> 
> #include <stdio.h>
> #include <math.h>
> 
> int
> compare_doubles (const void *a, const void *b)
> {
>   const double *da = (const double *) a;
>   const double *db = (const double *) b;
> 
>   return (*da > *db) - (*da < *db);
> }
> 
> int main (void)
> {
>   double t[3] = { 1.0, NAN, 2.0 };
>   printf ("%d\n", compare_doubles(t+0, t+1));
>   printf ("%d\n", compare_doubles(t+1, t+2));
>   printf ("%d\n", compare_doubles(t+0, t+2));
>   return 0;
> }
> 
> which gives
> 
> 0
> 0
> -1
> 
> while the initial 0 0 implies a third 0 with a transitive comparator.
> 

I see this is an manual issue rather than a GNU 'extension' to qsort semantic.
And I think we should fix BZ#31322 by using a transitive comparison instead of
trying to support such cases.