[PATCH] Re: glibc segfault on "special" long double values is _ok_!?

[Jakub Jelinek <jakub at redhat dot com>]

On Wed, Jun 06, 2007 at 02:32:20PM +0200, Jim Meyering wrote:
> Did you mean to close this bug as "resolved/invalid"?
> 
>     printf crashes on some 'long double' values
>     http://sourceware.org/bugzilla/show_bug.cgi?id=4586
>     [BTW, it segfaults on i686 rawhide, even with no compiler options]
> 
> I'm interested, because I don't want my applications to segfault on such
> inputs.  Sure it may look a little far-fetched, but I think it's not.
> Imagine such a bit pattern being injected into a network data stream
> that is then printed as a long double.  Just printing an arbitrary
> "long double" should not make a server vulnerable to a DoS attack.
> 
> If glibc were to stay this way, I would feel obliged to make applications
> I care about pull in the gnulib replacement printf infrastructure even
> when my system provides the latest glibc.  That would be a shame.
> 
> It may well be that the current glibc behavior is not prohibited by
> any standard, but I think that "quality of implementation" concerns
> (not to mention a desire for robustness and security) would dictate a
> more manageable result.

We already special case e.g. 0x0000.80000000.00000000 (pseudo denormal
with implicit bit set), so I guess we can also special case this other
misdesigned FP type.  Handling it as normal zeros is easiest IMHO,
special casing these in printf_fp.c would be more costly.  And, people
can always use %La to find out what is real zero and what is pseudo zero.

Also, x86_64 and ia64 as they use the same long double format as i386
should IMHO use the same ldbl2mpn.c implementation, not the ldbl-96
one which is presumably meant for m68k.

That said, programs that pass arbitrary bit patterns read from an insecure
source to *printf and other functions are definitely
broken as floating point numbers, they can hit signalling NaNs etc.

2007-06-06  Jakub Jelinek  <jakub@redhat.com>

	BZ #4586
	* sysdeps/i386/ldbl2mpn.c (__mpn_extract_long_double): Treat pseudo-zeros
	as zero.
	* sysdeps/x86_64/ldbl2mpn.c: New file.
	* sysdeps/ia64/ldbl2mpn.c: New file.

--- libc/sysdeps/i386/ldbl2mpn.c.jj	2001-07-06 06:55:52.000000000 +0200
+++ libc/sysdeps/i386/ldbl2mpn.c	2007-06-06 17:38:17.000000000 +0200
@@ -1,4 +1,4 @@
-/* Copyright (C) 1995, 1996, 1997, 2000 Free Software Foundation, Inc.
+/* Copyright (C) 1995, 1996, 1997, 2000, 2007 Free Software Foundation, Inc.
    This file is part of the GNU C Library.
 
    The GNU C Library is free software; you can redistribute it and/or
@@ -19,7 +19,7 @@
 #include "gmp.h"
 #include "gmp-impl.h"
 #include "longlong.h"
-#include "ieee754.h"
+#include <ieee754.h>
 #include <float.h>
 #include <stdlib.h>
 
@@ -46,7 +46,7 @@ __mpn_extract_long_double (mp_ptr res_pt
 #elif BITS_PER_MP_LIMB == 64
   /* Hopefully the compiler will combine the two bitfield extracts
      and this composition into just the original quadword extract.  */
-  res_ptr[0] = ((unsigned long int) u.ieee.mantissa0 << 32) | u.ieee.mantissa1;
+  res_ptr[0] = ((mp_limb_t) u.ieee.mantissa0 << 32) | u.ieee.mantissa1;
   #define N 1
 #else
   #error "mp_limb size " BITS_PER_MP_LIMB "not accounted for"
@@ -109,6 +109,13 @@ __mpn_extract_long_double (mp_ptr res_pt
 	    }
 	}
     }
+  else if (u.ieee.exponent < 0x7fff
+#if N == 2
+	   && res_ptr[0] == 0
+#endif
+	   && res_ptr[N - 1] == 0)
+    /* Pseudo zero.  */
+    *expt = 0;
 
   return N;
 }
--- libc/sysdeps/x86_64/ldbl2mpn.c.jj	2007-06-06 17:50:03.000000000 +0200
+++ libc/sysdeps/x86_64/ldbl2mpn.c	2007-06-06 17:49:56.000000000 +0200
@@ -0,0 +1 @@
+#include "../i386/ldbl2mpn.c"
--- libc/sysdeps/ia64/ldbl2mpn.c.jj	2007-06-06 17:50:03.000000000 +0200
+++ libc/sysdeps/ia64/ldbl2mpn.c	2007-06-06 17:49:56.000000000 +0200
@@ -0,0 +1 @@
+#include "../i386/ldbl2mpn.c"


	Jakub