New GNU C Library (glibc) security flaw reported on 30 Jan 2024

[Siddhesh Poyarekar]

On 2024-01-31 11:23, Vincent Lefevre wrote:
> If this was intentional, users may already use such a nontransitive
> comparison function. If this was not intentional, this shows that
> the problem is not obvious and that users less experienced than the
> glibc developers may fall in such a trap. So, in addition to make
> the glibc manual clear on the subject, I think that the library
> should handle such cases gracefully rather than considering that
> such cases will never occur. For instance, this could be just an
> indeterminate ordering. Or if the goal is to warn the user about
> the issue (in case it is detected), an assertion failure instead
> of indeterminate ordering.
> 

The advisory[1] has a statement on behalf of the glibc security team, 
reproduced here for convenience:

~~~
This memory corruption in the GNU C Library through the qsort function 
is invoked by an application passing a non-transitive comparison 
function, which is undefined according to POSIX and ISO C standards.  As 
a result, we are of the opinion that the resulting CVE, if any, should 
be assigned to any such calling applications and subsequently fixed by 
passing a valid comparison function to qsort and not to glibc.  We 
however acknowledge that this is a quality of implementation issue and 
we fixed this in a recent refactor of qsort.  We would like to thank 
Qualys for sharing their findings and helping us validate our recent 
changes to qsort.
~~~

Hopefully this addresses your concern; the current glibc implementation 
*is* robust to these issues (the advisory explicitly calls that out) but 
the manual also needs to be fixed so that we don't give the incorrect 
impression that this is a GNU extension.

Thanks,
Sid

[1] https://www.qualys.com/2024/01/30/qsort.txt