exp/1978: >> operator uses sign of right-hand operand

[johnboy at moriancumer dot net]

>Number:         1978
>Category:       exp
>Synopsis:       >> operator uses sign of right-hand operand
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jul 29 00:48:01 UTC 2005
>Closed-Date:
>Last-Modified:
>Originator:     >> operator uses sign of right-hand operand
>Release:        6.3
>Organization:
>Environment:
Debian Linux (unstable) with custom kernel 2.6.10, running on an AMD Athlon 2600 CPU (32 bit).
gcc version 4.0.1 (Debian 4.0.1-2)
gdb version 6.3 (Debian 6.3-6)
>Description:
GDB and GCC employ different methods of evaluating expressions that use the >> operator when the right operand is of an unsigned type.

Consider the following table below. Each of the four expressions are evaluated by a small test program compiled with GCC, and also with GDB's print statement. The resulting evaluations are shown beside the expression. You'll note that they don't math in one case.
  GCC     GDB    Expression
 =====   =====  ========================================
  -1      -1     ~0 >> 24
  -1      255    ~0 >> (unsigned int) 24
  255     255    (unsigned int) ~0 >> 24
  255     255    (unsigned int) ~0 >> (unsigned int) 24

(~0) is the two's complement value for -1. If that value is considered signed (which it is by default), then GCC's right-shift operator correctly pads the field with '1's, such that (~0 >> 24) == (~0) == -1.

In each of the cases above, GCC's evaluation demonstrates that it doesn't care about the sign of the right operand, while GDB's result cares very much. For GCC, the sign of the value to shift is the same as that of the left operand. 

For GDB, on the other hand, the shift value becomes unsigned if either the left *or* right operand is unsigned, instead of just depending on the left operand as one would expect.
>How-To-Repeat:
The attached file includes a simple C program that demonstrates the GCC output in the table above. To see the GDB output, start GDB, then type "print ~0 >> 24" and press enter, substituting in each of the above expressions in order.
>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted:
----gnatsweb-attachment----
Content-Type: application/octet-stream; name="test.c"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="test.c"

LyogdGVzdC5jIAogKiAgIERlbW9uc3RyYXRlcyBob3cgYSBwcm9ncmFtIGNvbXBpbGVkIGJ5IEdD
QyBoYW5kbGVzIHNpZ24gaW4gdGhlID4+IG9wZXJhdG9yLgogKi8KCiNpbmNsdWRlIDxzdGRpby5o
PgoKaW50IG1haW4odm9pZCkgewogICB1bnNpZ25lZCBpbnQgbWFzazsKCiAgIHByaW50ZigiVGVz
dCAxOiAlZFxuIiwgICggICAgICAgICAgICAgICB+MCkgPj4gICAgICAgICAgICAgICAgMjQgICAp
OwogICBwcmludGYoIlRlc3QgMjogJWRcbiIsICAoICAgICAgICAgICAgICAgfjApID4+ICh1bnNp
Z25lZCBpbnQpIDI0ICAgKTsKICAgcHJpbnRmKCJUZXN0IDM6ICVkXG4iLCAgKCh1bnNpZ25lZCBp
bnQpIH4wKSA+PiAgICAgICAgICAgICAgICAyNCAgICk7CiAgIHByaW50ZigiVGVzdCA0OiAlZFxu
IiwgICgodW5zaWduZWQgaW50KSB+MCkgPj4gKHVuc2lnbmVkIGludCkgMjQgICApOwoKICAgcmV0
dXJuIDA7Cn0KCi8qIApjb21waWxlIGNvbW1hbmQ6CiAgZ2NjIC1vIHRlc3QgdGVzdC5jCkhlcmUg
aXMgdGhlIG91dHB1dDoKPT09PT09PT09PT0KVGVzdCAxOiAtMQpUZXN0IDI6IC0xClRlc3QgMzog
MjU1ClRlc3QgNDogMjU1Cj09PT09PT09PT09CiovCg==