Re: RFC: Longjmp vs LD_POINTER_GUARD revisited

[Daniel Jacobowitz <drow at false dot org>]

On Mon, Nov 16, 2009 at 02:53:50PM +0000, Pedro Alves wrote:
> On Monday 16 November 2009 14:36:13, Daniel Jacobowitz wrote:
> > On Sun, Nov 15, 2009 at 03:05:33PM -0800, Paul Pluzhnikov wrote:
> > > Still it's trivial to discover the canary without disassembling
> > > anything (disassembling requires symbols, which may be stripped):
> > > there are only 3 different algorithms I've seen (no canary, XOR,
> > > XOR+shift-by-9). Hmm, looks like x86_64 has XOR+shift-by-17 now, but
> > > ia64, SPARC and PPC all have just "plain XOR".
> > 
> > I don't know about "trivial" - could you explain how you would do this
> > without disassembling?  I think that at the least we'd have to call
> > setjmp in the inferior, which has risks with signals / multiple
> > threads / etc.  I don't think we call functions in the inferior for
> > discovery otherwise.
> 
> I've had this patch below laying here for ages, that does
> what Paul is suggesting, while trying to avoid infcalls.
> It's quite ugly...  I've been meaning to post this for ages,
> but I'm a bit embarrassed by the hack :-)  It doesn't work in
> every possible case.

So rather than disassembling setjmp to see what it is, you try various
algorithms and then scan backwards looking for a call to setjmp.  If
you find one, you've succeeded.  Sounds like it will work in an
acceptable set of cases.

> There are two issues that made me try this route:
> 
>  - single-stepping all the way through longjmp turned out
>    to be slow.  Did you not perceive that too?  Maybe we
>    could limit the amount of single-stepping by not placing
>    the longjmp breakpoint at 'longjmp', but at one of its callees
>    that's closer to the real long jump.

I've only been testing on x86_64-linux native so far, where the speed
is not as measurable.  It didn't seem to be too many instructions.

>  - I've tried a similar hack as yours and found that I had
>    to add several more "functions still within longjmp" special
>    cases.  I had tried it on netbsd and Windows too.  NetBSD
>    was also bad at needing special casing.  I'll post the relevant
>    bits of the patch in a bit, when I find it :-)

I don't quite follow your patch... I'd have to sit down and stare at
it for a while.

So we have three strategies:

* Step.  Various hacks, reconcile your patch and mine (probably mostly
by taking yours), test on a lot of platforms.

* Infcall setjmp to examine its behavior.  Risky in that infcalls are
always risky.  I feel like I don't have good insight into all the ways
this could go wrong.

* Try multiple algorithms in the target on platforms with jmp_buf
encryption.

You've explored these more than I have.  Which one do you like best?
If it's the last one, then I should pick up your attached patch... I
don't think it's too awful.  The only thing I don't care for is
the target method for the thread area.  I'd rather make the value
available through the regcache.  IIRC we were going to call this
%fs_base at some point.

-- 
Daniel Jacobowitz
CodeSourcery