This is the mail archive of the mailing list for the glibc project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Ruby testsuite failures because of pointer mangling on 32-bit ARM?

On 12 December 2013 21:33, Carlos O'Donell <> wrote:
> On 12/12/2013 01:20 PM, David Miller wrote:
>> From: David Miller <>
>> Date: Thu, 12 Dec 2013 12:14:41 -0500 (EST)
>>> From: "Carlos O'Donell" <>
>>> Date: Thu, 12 Dec 2013 00:55:01 -0500
>>>> It treats the jmp_buf as an array of VALUE sized pointers that
>>>> it can examine to determine if there are pointers to the heap.
>>> Sounds similar to what any other garbage collector will do, scan
>>> the processes address space looking for pointers.
>>> I'm pretty sure Boehm-GC does something similar, although perhaps
>>> it scans the entire process stack from the point in which it is
>>> called instead of using jmpbuf's to delineate spans of stack
>>> areas like Ruby does.
>> And, indirectly, realize that even a straight stack scan is going
>> to potentially break if you start mangling pointers in jmpbuf.
>> Consider the case where if the jmpbuf is on the processes stack, and
>> normally it would get scanned by GC and the pointer followed to find
>> memory references, and now that would not work because the pointer is
>> mangled.
> Only if the jpmbuf pointer is the only way to access the other objects.
> It seems like ruby is optimizing by starting with the jmpbuf to look
> for object pointers.
>> I think all of these schemes are legitimate and erroneously broken
>> by pointer mangling.
> The problem I have is that I can't objectively evaluate how well
> pointer mangling is working as a security feature because I lack
> any data about it. Therefore I can't compare it to the pain it would
> cause ruby to reimplement GC using some other more costly scanning.
> What do we do here?

If the frame pointer register is being used as a frame pointer it
can't contain a data pointer. Likewise the stack pointer and lr. As
far as i can tell the only issue is encrypting the frame pointer in
the -fomit-frame-pointer case.

Did the patch I submitted fix the ruby testsuite issue?

Will Newton
Toolchain Working Group, Linaro

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]