[binutils-gdb] Fix malloc allocation size sanity check

[Luis Machado]

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=6d8a0a5e90936d4bea9bf1ce9b4e1c22d9aaccae

commit 6d8a0a5e90936d4bea9bf1ce9b4e1c22d9aaccae
Author: Luis Machado <luis.machado@linaro.org>
Date:   Wed Aug 12 17:02:32 2020 -0300

    Fix malloc allocation size sanity check
    
    During debugging of PR26362, it was noticed that the malloc size check
    in check_type_length_before_alloc wasn't detecting an allocation attempt
    of a huge amount of bytes, making GDB run into an internal error.
    
    This happens because we're using an int to store a type's length. When the
    type length is large enough, the int will overflow and the max_value_size
    check won't work anymore.
    
    The following patch fixes this by making the length variable a ULONGEST.
    
    Printing statements were also updated to show the correct number of bytes.
    
    gdb/ChangeLog:
    
    2020-08-12  Luis Machado  <luis.machado@linaro.org>
    
            * value.c (check_type_length_before_alloc): Use ULONGEST to store a
            type's length.
            Use %s and pulongest to print the length.

Diff:
---
 gdb/ChangeLog |  6 ++++++
 gdb/value.c   | 10 +++++-----
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 4abc47dab42..27c6849cd6a 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,3 +1,9 @@
+2020-08-12  Luis Machado  <luis.machado@linaro.org>
+
+	* value.c (check_type_length_before_alloc): Use ULONGEST to store a
+	type's length.
+	Use %s and pulongest to print the length.
+
 2020-08-12  Pedro Alves  <palves@redhat.com>
 
 	* NEWS: Move "Multi-target debugging support" item to the
diff --git a/gdb/value.c b/gdb/value.c
index aac9baaaf56..a6e21309f85 100644
--- a/gdb/value.c
+++ b/gdb/value.c
@@ -997,16 +997,16 @@ show_max_value_size (struct ui_file *file, int from_tty,
 static void
 check_type_length_before_alloc (const struct type *type)
 {
-  unsigned int length = TYPE_LENGTH (type);
+  ULONGEST length = TYPE_LENGTH (type);
 
   if (max_value_size > -1 && length > max_value_size)
     {
       if (type->name () != NULL)
-	error (_("value of type `%s' requires %u bytes, which is more "
-		 "than max-value-size"), type->name (), length);
+	error (_("value of type `%s' requires %s bytes, which is more "
+		 "than max-value-size"), type->name (), pulongest (length));
       else
-	error (_("value requires %u bytes, which is more than "
-		 "max-value-size"), length);
+	error (_("value requires %s bytes, which is more than "
+		 "max-value-size"), pulongest (length));
     }
 }