pointer comparison breaks Solaris' qsort
Alexandre Oliva
aoliva@redhat.com
Tue Jun 8 19:46:00 GMT 2004
On Jun 8, 2004, Jakub Jelinek <jakub@redhat.com> wrote:
> On Tue, Jun 08, 2004 at 06:53:31AM -0300, Alexandre Oliva wrote:
>> This patch fixes a linker crash caused by inconsistent results being
>> returned by the function passed to qsort(). In certain cases, when it
>> returns the same value when comparing both a with b and b with a, the
>> qsort function may end up being called with a pointer that's not
>> within the bounds of the array passed to qsort().
> That is a Solaris bug then.
Nope. It's not the difference between the arguments passed to the
compare function that isn't a multiple of the pointer size. We pass
qsort() an array of pointers, and it's the pointers we get from the
array (not pointers into the array) that we use in a pointer
subtraction, even though the objects they point to are not part of the
same array.
So we invoke undefined behavior twice: by computing the difference
between pointers to different objects, and then by returning
inconsistent values from the compare function because the differences
were miscomputed. As soon as you invoke undefined behavior, all bets
are off.
--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}
More information about the Binutils
mailing list