This is the mail archive of the mailing list for the glibc project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Ruby testsuite failures because of pointer mangling on 32-bit ARM?

On 12/12/2013 04:46 PM, Will Newton wrote:
> On 12 December 2013 21:33, Carlos O'Donell <> wrote:
>> On 12/12/2013 01:20 PM, David Miller wrote:
>>> From: David Miller <>
>>> Date: Thu, 12 Dec 2013 12:14:41 -0500 (EST)
>>>> From: "Carlos O'Donell" <>
>>>> Date: Thu, 12 Dec 2013 00:55:01 -0500
>>>>> It treats the jmp_buf as an array of VALUE sized pointers that
>>>>> it can examine to determine if there are pointers to the heap.
>>>> Sounds similar to what any other garbage collector will do, scan
>>>> the processes address space looking for pointers.
>>>> I'm pretty sure Boehm-GC does something similar, although perhaps
>>>> it scans the entire process stack from the point in which it is
>>>> called instead of using jmpbuf's to delineate spans of stack
>>>> areas like Ruby does.
>>> And, indirectly, realize that even a straight stack scan is going
>>> to potentially break if you start mangling pointers in jmpbuf.
>>> Consider the case where if the jmpbuf is on the processes stack, and
>>> normally it would get scanned by GC and the pointer followed to find
>>> memory references, and now that would not work because the pointer is
>>> mangled.
>> Only if the jpmbuf pointer is the only way to access the other objects.
>> It seems like ruby is optimizing by starting with the jmpbuf to look
>> for object pointers.
>>> I think all of these schemes are legitimate and erroneously broken
>>> by pointer mangling.
>> The problem I have is that I can't objectively evaluate how well
>> pointer mangling is working as a security feature because I lack
>> any data about it. Therefore I can't compare it to the pain it would
>> cause ruby to reimplement GC using some other more costly scanning.
>> What do we do here?
> If the frame pointer register is being used as a frame pointer it
> can't contain a data pointer. Likewise the stack pointer and lr. As
> far as i can tell the only issue is encrypting the frame pointer in
> the -fomit-frame-pointer case.
> Did the patch I submitted fix the ruby testsuite issue?

I'm still verifying this. Perhaps wait to checkin the patch until I give
an all clear?


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]