[binutils-gdb] Avoid undefined behavior in parse_number

[Tom Tromey]

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=20562150d8a894bc91657c843ee88c508188e32e

commit 20562150d8a894bc91657c843ee88c508188e32e
Author: Tom Tromey <tom@tromey.com>
Date:   Sun Jul 29 20:23:33 2018 -0600

    Avoid undefined behavior in parse_number
    
    -fsanitize=undefined pointed out that c-exp.y relied on undefined
    behavior here:
    
          if (c != 'l' && c != 'u')
    	n *= base;
    
    ...when a large hex constant "just fit" into a LONGEST, causing the
    high bit to be set.
    
    This fixes the problem by having the function work in an unsigned
    type.
    
    gdb/ChangeLog
    2018-10-03  Tom Tromey  <tom@tromey.com>
    
    	* c-exp.y (parse_number): Work in unsigned.  Remove casts.

Diff:
---
 gdb/ChangeLog |  4 ++++
 gdb/c-exp.y   | 10 ++++------
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 4be3033..97ee73b 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,5 +1,9 @@
 2018-10-03  Tom Tromey  <tom@tromey.com>
 
+	* c-exp.y (parse_number): Work in unsigned.  Remove casts.
+
+2018-10-03  Tom Tromey  <tom@tromey.com>
+
 	* dwarf2read.c (read_subrange_type): Make "negative_mask"
 	unsigned.
 
diff --git a/gdb/c-exp.y b/gdb/c-exp.y
index 0326ee0..09e31d2 100644
--- a/gdb/c-exp.y
+++ b/gdb/c-exp.y
@@ -1760,10 +1760,8 @@ static int
 parse_number (struct parser_state *par_state,
 	      const char *buf, int len, int parsed_float, YYSTYPE *putithere)
 {
-  /* FIXME: Shouldn't these be unsigned?  We don't deal with negative values
-     here, and we do kind of silly things like cast to unsigned.  */
-  LONGEST n = 0;
-  LONGEST prevn = 0;
+  ULONGEST n = 0;
+  ULONGEST prevn = 0;
   ULONGEST un;
 
   int i = 0;
@@ -1922,7 +1920,7 @@ parse_number (struct parser_state *par_state,
 	 on 0x123456789 when LONGEST is 32 bits.  */
       if (c != 'l' && c != 'u' && n != 0)
 	{	
-	  if ((unsigned_p && (ULONGEST) prevn >= (ULONGEST) n))
+	  if (unsigned_p && prevn >= n)
 	    error (_("Numeric constant too large."));
 	}
       prevn = n;
@@ -1940,7 +1938,7 @@ parse_number (struct parser_state *par_state,
      the case where it is we just always shift the value more than
      once, with fewer bits each time.  */
 
-  un = (ULONGEST)n >> 2;
+  un = n >> 2;
   if (long_p == 0
       && (un >> (gdbarch_int_bit (parse_gdbarch (par_state)) - 2)) == 0)
     {