Re: pointer comparison breaks Solaris' qsort

[Jakub Jelinek <jakub at redhat dot com>]

On Tue, Jun 08, 2004 at 06:53:31AM -0300, Alexandre Oliva wrote:
> This patch fixes a linker crash caused by inconsistent results being
> returned by the function passed to qsort().  In certain cases, when it
> returns the same value when comparing both a with b and b with a, the
> qsort function may end up being called with a pointer that's not
> within the bounds of the array passed to qsort().

That is a Solaris bug then.
http://www.opengroup.org/onlinepubs/009695399/functions/qsort.html
is clear on this, for each argument p to the compar function:
((char *)p - (char *)base) % width == 0
(char *)p >= (char *)base
(char *)p < (char *)base + nel * width

Similarly ISO C99, 7.20.5:
The implementation shall ensure that the second argument of the comparison
function (when called from bsearch), or both arguments (when called from
qsort), are pointers to elements of the array.

That is, if the value passed is p, then the following expressions are always
nonzero:
((char *)p - (char *)base) % size == 0
(char *)p >= (char *)base
(char *)p < (char *)base + nmemb * size

This doesn't mean ld doesn't have to work around Solaris bugs, just thought
I'd mention this.

	Jakub