This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
[PATCH] Fix typing of the bit twiddling done in_dl_setup_stack_chk_guard.
- From: David Miller <davem at davemloft dot net>
- To: libc-alpha at sourceware dot org
- Date: Sun, 11 Mar 2012 19:47:51 -0700 (PDT)
- Subject: [PATCH] Fix typing of the bit twiddling done in_dl_setup_stack_chk_guard.
I noticed a warning that triggers on big-endian 64-bit platforms.
Basically:
(0xff << (8 * (sizeof (ret) - 1)))
Doesn't work because 0xff is an "unsigned int" (which is 32-bit) and
we're shifting up into the topmost byte of "ret" based upon it's size
which is "sizeof(uintptr_t)" which is 64-bit.
I used a cast to get the typing right, and I added a cast to the
little-endian case for consistency, even though it isn't strictly
necessary.
Just out of curiosity I took a look at how GCC handles this undefined
situation using this code snippet:
--------------------
unsigned long bar(unsigned long x)
{
return x & (0xff << (8 * (sizeof(unsigned long) - 1)));
}
--------------------
For 64-bit sparc it essentially truncated the shift to 24-bits and
generated this:
mov -1, %g1
sllx %g1, 24, %g1
and %o0, %g1, %o0
And on x86-64 gcc emits "xorl %eax, %eax" :-)
Committed to master.
* sysdeps/unix/sysv/linux/dl-osinfo.h (_dl_setup_stack_chk_guard):
Fix masking out of the most significant byte of random value used.
---
ChangeLog | 3 +++
sysdeps/unix/sysv/linux/dl-osinfo.h | 4 ++--
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 85993dc..29b7c67 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,8 @@
2012-03-11 David S. Miller <davem@davemloft.net>
+ * sysdeps/unix/sysv/linux/dl-osinfo.h (_dl_setup_stack_chk_guard):
+ Fix masking out of the most significant byte of random value used.
+
* sysdeps/sparc/fpu/libm-test-ulps: Update.
2012-03-10 Andreas Schwab <schwab@linux-m68k.org>
diff --git a/sysdeps/unix/sysv/linux/dl-osinfo.h b/sysdeps/unix/sysv/linux/dl-osinfo.h
index 780b20a..1ff8a2f 100644
--- a/sysdeps/unix/sysv/linux/dl-osinfo.h
+++ b/sysdeps/unix/sysv/linux/dl-osinfo.h
@@ -95,9 +95,9 @@ _dl_setup_stack_chk_guard (void *dl_random)
directly and not use the kernel-provided data to seed a PRNG. */
memcpy (ret.bytes, dl_random, sizeof (ret));
#if BYTE_ORDER == LITTLE_ENDIAN
- ret.num &= ~0xff;
+ ret.num &= ~(uintptr_t)0xff;
#elif BYTE_ORDER == BIG_ENDIAN
- ret.num &= ~(0xff << (8 * (sizeof (ret) - 1)));
+ ret.num &= ~((uintptr_t)0xff << (8 * (sizeof (ret) - 1)));
#else
# error "BYTE_ORDER unknown"
#endif
--
1.7.6.401.g6a319