strtoul(-1) discussion

[Florian Weimer]

On 04/07/2013 03:20 PM, Fabrice Bauzac wrote:
> The algorithm behind GNU libc's strtoul seems to be:
>
>    * store the sign independently from the digits.
>    * perform a strtoul on the digits without the sign, producing ulong
>      value N.
>    * if the sign is negative, perform N := -N.
>    * return N.
>
> The problem is that [1] says:
>
>    "If the correct value is outside the range of representable values,
>    {ULONG_MAX} or {ULLONG_MAX} shall be returned and errno set to
>    [ERANGE]."


POSIX 2008 also says: "If the subject sequence begins with a minus-sign, 
the value resulting from the conversion shall be negated."  I think that 
means that POSIX actually mandates the (admittedly weird) glibc 
behavior.  The C and POSIX committees probably couldn't bring themselves 
to clearly specify this behavior, and that's where the ambiguity comes from.

It's even clearer for the %u conversion specifier, where this behavior 
is pretty explicitly mandated.

> I have made some experiments on Solaris, and the Solaris
> implementation does not follow its specification.  The behaviour of
> Solaris's strtoul seems to be the same as GNU libc's strtoul!

That's actually what I expect—GNU libc's behavior matches that of legacy 
systems.

There are some other non-GNU resources which describe the odd behavior, 
e.g.:

"This function returns an unsigned long int. If the given string starts 
with a minus sign, the result is a negative number that's cast to be 
unsigned."

<http://www.users.pjwstk.edu.pl/~jms/qnx/help/watcom/clibref/src/strtoul.html>

(The page source dates this to 1999, so it could have been influenced by 
glibc, but it's somewhat unlikely because GNU/Linux wouldn't have been 
considered the gold standard back then. And the QNX folks probably don't 
even today.)

There's an old discussion on comp.std.c dating back to September 1992, 
"strtoul on negative numbers".  It strongly suggests that back then, 
everyone implemented the strange behavior.

I guess we're stuck with it.

-- 
Florian Weimer / Red Hat Product Security Team