regression with huge integer

[Jeff Johnston]

Andrew Cagney wrote:
>> gdb.stabs/weird.def has this huge integer:
>>
>>   # 256-bit integer.  The point is obviously not that GDB should have a
>>   # special case for this size, but that an integer of any size should
>>   # work (at least for printing in hex, not necessarily for 
>> arithmetic.    .stabs "int256var:G203=bu32;0;256;", N_GSYM,0,0, 0
>>   # The value is palindromic, so it works whether words are big or little
>>   # endian.    .globl int256var
>>   .data
>>   .align_it
>>        int256var:
>>   .long 42
>>        .long 0x2b, 0x2c, 0x2d, 0x2d, 0x2c, 0x2b, 0x2a
>>
>> gdb 6.0 can print this just fine:
>>
>>   (gdb) print int256var
>>   $1 = 0x0000002a0000002b0000002c0000002d0000002d0000002c0000002b0000002a
> 
> 
> That should be decimal :-/
> 
>>   (gdb) print /x int256var
>>   $2 = 0x0000002a0000002b0000002c0000002d0000002d0000002c0000002b0000002a
>>
>> But with the recent simplification to print_scalar_formatted, gdb HEAD 
>> says:
>>
>>   (gdb) print int256var
>>   $1 = 0x0000002a0000002b0000002c0000002d0000002d0000002c0000002b0000002a
>>   (gdb) print /x int256var
>>   $2 = That operation is not available on integers of more than 8 bytes.
>>
>> This causes a regression in the test results.
> 
> 
> Hmm, two steps forward, one step back.
> 
>> I would like to just accept this output and change the test script.
>> Specifically, in gdb.stabs/weird.exp:
>>
>>       # This big number needs to be kept as one piece
>>   -     gdb_test "p/x int256var" " = 
>> 0x0*2a0000002b0000002c0000002d0000002d0000002c0000002b0000002a" "print 
>> very big integer"
>>   +     gdb_test "print int256var" " = 
>> 0x0*2a0000002b0000002c0000002d0000002d0000002c0000002b0000002a" "print 
>> very big integer"
>>     Is this a good idea?  Or should I file a bug that "print /x" does 
>> not work
>> in this case?
> 
> 
> Jeff and I looked at the problem.
> 
> Given some sort of very large scalar _and_ a scalar format, I think GDB 
> can correctly print it.  Looking at the old 60 code, this:
> 
>   if (len > sizeof (LONGEST)
>       && (format == 't'
>           || format == 'c'
>           || format == 'o'
>           || format == 'u'
>           || format == 'd'
>           || format == 'x'))
>     {
>       if (!TYPE_UNSIGNED (type)
>           || !extract_long_unsigned_integer (valaddr, len, &val_long))
>         {
>           /* We can't print it normally, but we can print it in hex.
>              Printing it in the wrong radix is more useful than saying
>              "use /x, you dummy".  */
>           /* FIXME:  we could also do octal or binary if that was the
>              desired format.  */
>           /* FIXME:  we should be using the size field to give us a
>              minimum field width to print.  */
> 
>           if (format == 'o')
>             print_octal_chars (stream, valaddr, len);
>           else if (format == 'd')
>             print_decimal_chars (stream, valaddr, len);
>           else if (format == 't')
>             print_binary_chars (stream, valaddr, len);
>           else
>             /* replace with call to print_hex_chars? Looks
>                like val_print_type_code_int is redoing
>                work.  - edie */
> 
>             val_print_type_code_int (type, valaddr, stream);
> 
> would just need to be seriously reduced to something like:
> 
>     if (len > sizeof (LONGEST)
>         && some sort of scalar (TYPE)
>         && some sort of scalar (FORMAT))
>       if (format == ..)   
>         print_FORMAT_chars (...);
>       ...
>       else if (format == 'x')
>          print_hex_chars (...);
>           else
>         we've botched it -- don't call val_print_type_code_int
> 
> where each format is explicitly handled.      ...
> 
> The only one that appears to be missing is 'c', and there something very 
> similar to print_hex_chars would do the trick (using LA_EMIT_CHAR).
> 
> It might even, eventually, be possible to simplify this code to the 
> point where all scalar formatted scalars are always printed directly 
> from their byte buffer (no unpack longest call).
> 
> Andrew
> 
> 
> 

I'll start working on a patch and submit it for review.

-- Jeff J.