vfprintf typing problem

[Paul Eggert]

On 03/28/2012 03:00 PM, David Miller wrote:
> Another option is to keep the type as 'int', but abort with an error
> earlier, say as soon as the sign bit is set in the width.

At the risk of repeating the obvious, in general one cannot
reliably test for signed integer overflow by looking at the
sign bit, since in C the behavior is undefined after integer
overflow.  Assuming these declarations:

   int a = [some nonnegative value];
   int b = [some nonnegative value];

then the following C code is not portable:

   int sum = a + b;
   if (sum < 0)
     return OVERFLOW;

since GCC and other compilers may optimize away the last two lines.

Here is one way to work around the problem:

   if (INT_MAX - b < a)
     return OVERFLOW;
   int sum = a + b;

Gnulib's intprops module has some macros designed to help
make this a bit easier, e.g.:

   if (INT_ADD_OVERFLOW (a, b))
     return OVERFLOW;
   int sum = a + b;

The intprops module could be used in glibc internals if that'd be helpful.
For more, please see its documentation, starting at
<http://www.gnu.org/software/gnulib/manual/html_node/Integer-Properties.html>.