Re: uprobes-via-utrace notes

[Roland McGrath <roland at redhat dot com>]

Jim had a brief opportunity to look at utrace before leaving on vacation
and sent me some questions privately.  I thought the details that arise
here might be worthwhile for others to see.  I won't comment on Jim's
uprobes ideas directly, as he has only begun to develop them.  Here are
some clarifications about details of utrace in response to some of Jim's
remarks.

> When the breakpoint is hit, kprobes shows no interest, and do_trap()
> handles it.  utrace calls uprobes's report_signal callback with
> action = SIGNAL_CORE and info.si_signo = SIGTRAP.  (If there's a gdb
> breakpoint there, it looks like action could be SIGNAL_IGN by the
> time uprobes sees it.  TBD)  

That's correct.  The way that multiple engines interact is something of an
unresolved area in utrace.  As things stand, if ptrace was in the list
first then it will get the callback first, tell the ptracer about it, and
change the action to UTRACE_SIGNAL_IGN.  A "noninvasive" tracing engine
would want to be the one to go first, and hide the signal from other engines.
There isn't currently any good way to control the order of the list.

> - Saves the regs pointer (since we'll need it, I think, and it's not
> passed to report_quiesce).

It is not a kosher part of the interface as defined heretofore to store
this, though I think in fact it would work ok.  So I'm unsure whether to
object to this or not.  I've expected that engines would use the
utrace_regset interface exclusively for any asynchronous access to thread
registers (asynchronous meaning not made from a report_* callback itself).

> - Returns (UTRACE_ACTION_IGN | UTRACE_ACTION_QUIESCE | UTRACE_ACTION_NEWSTATE)
> so that the SIGTRAP won't be delivered, and the process will become
> quiescent (and our report_quiesce handler will be called).

That is what will happen.

> Our report_quiesce callback runs the user-specified handler.  We want
> to be quiescent so the handler has more leeway about sleeping, etc.

This is incorrect.  Your callbacks should never sleep arbitrarily.  You
don't get to completely hijack the thread at any point.  Instead, by
keeping UTRACE_ACTION_QUIESCE set, you ensure that it does not resume in
user mode until you say it should.  

> Our report_quiesce callback also needs to set single-stepping in
> motion.  If we single-step inline, we need to poke the opcode into
> the instruction stream and (for i386 and x86_64) adjust the IP back
> to the beginning of the instruction.  

Not viable for multithreaded, as has been discussed in the past.
But that is not an issue of understanding the utrace interface.
So let's leave this for later discussion.

> I'm not too clear on the "approved" way of doing this.  It appears that
> utrace_regset() can be used to get and set the saved registers.  

That's correct.  utrace_regset is the most thoroughly kosher way to get at
a thread's user registers.

> Or can we just access pt_regs* directly?

Inside a callback that takes struct pt_regs * as an argument, sure you can.

> Use access_process_vm() to peek/poke memory.

Correct.

> Our report_quiesce callback enables single-stepping by returning
> (UTRACE_ACTION_SINGLESTEP | UTRACE_ACTION_NEWSTATE), or something
> like that.

Correct.

> After the instruction is single-stepped, our report_signal callback
> is called with action = UTRACE_SIGNAL_HANDLER.  In this case, the
> info, orig_ka, and return_ka args are NULL; and "no signal will
> (ever) actually be delivered regardless of the return value."

This is incorrect.  The UTRACE_SIGNAL_HANDLER value is a special case only
when you single-step into a signal handler.  It does not apply to all
single-stepping.  After stepping an instruction you just get a SIGTRAP or
whatever the arch code generates.  When you resume with single-stepping and
an unblocked, caught signal is pending (e.g. PTRACE_SINGLESTEP with a
nonzero signal number argument)q, no user instruction is actually executed.
Instead, the signal frame is set up and the PC changed to the first
instruction of the user handler specified with sigaction.  Before executing
the first instruction of the handler, the thread stops and makes a
report_signal callback with UTRACE_SIGNAL_HANDLER.  This obscure special
case is the only thing that code relates to.

> report_signal will need to put back the breakpoint instruction
> (if we're single-stepping inline) or do the usual kprobes-like
> resume_execution stuff and re-adjust the IP (if out-of-line).
> Nothing needs to be done to turn off single-stepping.

This is incorrect.  All of the UTRACE_ACTION_STATE_MASK bits are state flags.
They remain in force until reset.  To resume normally when previously you
were single-stepping, you need to use utrace_set_flags or have a callback
return UTRACE_ACTION_NEWSTATE | UTRACE_ACTION_RESUME.


Thanks,
Roland