regression with huge integer

[Andrew Cagney]

> gdb.stabs/weird.def has this huge integer:
> 
>   # 256-bit integer.  The point is obviously not that GDB should have a
>   # special case for this size, but that an integer of any size should
>   # work (at least for printing in hex, not necessarily for arithmetic.  
>   .stabs "int256var:G203=bu32;0;256;", N_GSYM,0,0, 0
>   # The value is palindromic, so it works whether words are big or little
>   # endian.  
>   .globl int256var
>   .data
>   .align_it
>        int256var:
>   .long 42
>        .long 0x2b, 0x2c, 0x2d, 0x2d, 0x2c, 0x2b, 0x2a
> 
> gdb 6.0 can print this just fine:
> 
>   (gdb) print int256var
>   $1 = 0x0000002a0000002b0000002c0000002d0000002d0000002c0000002b0000002a

That should be decimal :-/

>   (gdb) print /x int256var
>   $2 = 0x0000002a0000002b0000002c0000002d0000002d0000002c0000002b0000002a
> 
> But with the recent simplification to print_scalar_formatted, gdb HEAD says:
> 
>   (gdb) print int256var
>   $1 = 0x0000002a0000002b0000002c0000002d0000002d0000002c0000002b0000002a
>   (gdb) print /x int256var
>   $2 = That operation is not available on integers of more than 8 bytes.
> 
> This causes a regression in the test results.

Hmm, two steps forward, one step back.

> I would like to just accept this output and change the test script.
> Specifically, in gdb.stabs/weird.exp:
> 
> 	  # This big number needs to be kept as one piece
>   - 	gdb_test "p/x int256var" " = 0x0*2a0000002b0000002c0000002d0000002d0000002c0000002b0000002a" "print very big integer"
>   + 	gdb_test "print int256var" " = 0x0*2a0000002b0000002c0000002d0000002d0000002c0000002b0000002a" "print very big integer"
>     
> Is this a good idea?  Or should I file a bug that "print /x" does not work
> in this case?

Jeff and I looked at the problem.

Given some sort of very large scalar _and_ a scalar format, I think GDB 
can correctly print it.  Looking at the old 60 code, this:

   if (len > sizeof (LONGEST)
       && (format == 't'
           || format == 'c'
           || format == 'o'
           || format == 'u'
           || format == 'd'
           || format == 'x'))
     {
       if (!TYPE_UNSIGNED (type)
           || !extract_long_unsigned_integer (valaddr, len, &val_long))
         {
           /* We can't print it normally, but we can print it in hex.
              Printing it in the wrong radix is more useful than saying
              "use /x, you dummy".  */
           /* FIXME:  we could also do octal or binary if that was the
              desired format.  */
           /* FIXME:  we should be using the size field to give us a
              minimum field width to print.  */

           if (format == 'o')
             print_octal_chars (stream, valaddr, len);
           else if (format == 'd')
             print_decimal_chars (stream, valaddr, len);
           else if (format == 't')
             print_binary_chars (stream, valaddr, len);
           else
             /* replace with call to print_hex_chars? Looks
                like val_print_type_code_int is redoing
                work.  - edie */

             val_print_type_code_int (type, valaddr, stream);

would just need to be seriously reduced to something like:

	if (len > sizeof (LONGEST)
	    && some sort of scalar (TYPE)
	    && some sort of scalar (FORMAT))
	  if (format == ..)	
	    print_FORMAT_chars (...);
	  ...
	  else if (format == 'x')
  	    print_hex_chars (...);
     	  else
	    we've botched it -- don't call val_print_type_code_int

where each format is explicitly handled.	  ...

The only one that appears to be missing is 'c', and there something very 
similar to print_hex_chars would do the trick (using LA_EMIT_CHAR).

It might even, eventually, be possible to simplify this code to the 
point where all scalar formatted scalars are always printed directly 
from their byte buffer (no unpack longest call).

Andrew