Ruby testsuite failures because of pointer mangling on 32-bit ARM?

[Carlos O'Donell]

On 12/10/2013 09:19 PM, David Miller wrote:
> From: Andrew Pinski <pinskia@gmail.com>
> Date: Tue, 10 Dec 2013 17:13:30 -0800
> 
>> On Tue, Dec 10, 2013 at 2:36 PM, David Miller <davem@davemloft.net> wrote:
>>> From: pinskia@gmail.com
>>> Date: Tue, 10 Dec 2013 11:57:03 -0800
>>>
>>>> Hmm, shouldn't ruby be using makecontext/restore context instead of
>>>> setjmp/longjmp for this purpose?  I think we should declare this as
>>>> a bug in ruby.
>>>
>>> I honestly think that what Ruby is doing is quite reasonable.
>>
>> How so, jmp_buf is supposed to be an opaque structure.  If a program
>> depends on the context of an opaque structure, then it is a bug in the
>> program rather than the library which changes what the opaque
>> structure contains.
> 
> Addressing jmpbuf specifically, changing jmpbuf's layout would break
> every GDB binary out there.
> 
> There are countless programs which want to see the frame pointer and
> use jmpbuf plus target-specific implementations to fetch that value.
> 
> But that's not what Ruby is doing here, it is copying the stack frame
> of a thread into another location and then using it in some way.  In
> fact I do not see any code in Ruby that actually looks into the jmpbuf
> contents at all.

I think it does. As Will pointed out:

In vm_core.h:

   474      jmp_buf machine_regs;

In vm.c:

  1589          if (GET_THREAD() != th && th->machine_stack_start &&
th->machine_stack_end) {
  1590              rb_gc_mark_machine_stack(th);
  1591              rb_gc_mark_locations((VALUE *)&th->machine_regs,
  1592                                   (VALUE *)(&th->machine_regs) +
  1593                                   sizeof(th->machine_regs) /
sizeof(VALUE));
  1594          }

In gc.c:

rb_gc_mark_locations -> gc_mark_locations -> mark_locations_array

2316 static void
2317 mark_locations_array(rb_objspace_t *objspace, register VALUE *x, register long n)
2318 {
2319     VALUE v;
2320     while (n--) {
2321         v = *x;
2322         (void)VALGRIND_MAKE_MEM_DEFINED(&v, sizeof(v));
2323         if (is_pointer_to_heap(objspace, (void *)v)) {
2324             gc_mark(objspace, v);
2325         }
2326         x++;
2327     }
2328 }

It treats the jmp_buf as an array of VALUE sized pointers that
it can examine to determine if there are pointers to the heap.

This kind of analysis seems invalid under jmp_buf pointer mangling.
I don't see how this code can possibly be correct.

It still doesn't fix the fact that glibc broke ruby, and we either
get to be jerks about it or be reasonable and not encrypt all the
pointers until we can get upstream ruby fixed.

What I don't understand is why this doesn't break other targets
that use mangling...

Cheers,
Carlos.