This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: utrace-based uprobes
On Fri, 2007-03-16 at 17:28 +0100, Mark Wielaard wrote:
> Hi,
>
> On Thu, 2007-03-15 at 17:21 -0700, Jim Keniston wrote:
> > o As of now, uprobes aren't inherited across fork()s.
>
> Is there a special reason for this? It looks like you currently do extra
> work in case of a fork() to remove the breakpoints, which most likely
> will be cleared anyway as soon as the thread does an exec().
The main reason is that a u[ret]probe object can be associated with only
one process at a time. We could clone the u[ret]probe objects for the
new process, but it's not clear how those new objects' addresses would
be communicated back to the instrumentation module. I'm sure it's
doable; it's just not high on our list. Suggestions welcome.
>
> > o Probe registration and ungregistration in the context of a
> > multithreaded application, are asynchronous events. All threads
> > need to be QUIESCED before the program text is modified to insert
> > the breakpoint.
>
> Is this a requirement? I suppose modifying code while another thread can
> execute that code path is never safe. But do you have to suspend all
> threads? A suggestion would be to mark the code page as not executable
> so any thread that accesses that code path gets trapped till the code
> has been adjusted.
We settled on the quiesce-everybody approach a long time ago. If we
rejected your idea, I can't remember why. Maybe a colleague will inform
me. Otherwise, it seems worth investigating.
>
> > +When a CPU hits the breakpoint instruction, a trap occurs, the CPU's
> > +user-mode registers are saved, and a SIGTRAP signal is generated.
> > +Uprobes intercepts the SIGTRAP and finds the associated uprobe.
>
> One problem that I recently encountered is that the process might have
> SIGTRAP masked or you might insert and then hit a breakpoint while a
> SIGTRAP handler is running in the user process (which normally means the
> handler is masked and hitting the breakpoint will then most likely kill
> the process completely). I don't know a way around this. If you can
> detect such a case you would need to enter the kernel differently. On
> the other hand this might not be a very common case. But it might be
> something you want to keep in mind.
OK. At the very least, we'd need to document this as a limitation.
>
> > +If the number of times a function is called does not match the
> > +number of times it returns (e.g., if a function exits via longjmp()),
> > +registering a return probe on that function may produce undesirable
> > +results.
>
> What do these undesirable results include
Returning to the wrong address. For example, if A calls B, and you have
uretprobes on both A and B, and B longjmps back into A, B's return
address will be left atop that task's stack of return addresses. So
when A returns, it'll return using B's return address.
> (is it unsafe to add return
> probes)?
It's safe within the stated limitations. In particular, it's fine if a
process exits from within a uretprobed function.
> Is there a way to detect such functions/behavior?
Perhaps the uretprobe function-entry handler could remember the stack
pointer, and the trampoline (return) handler could perform an
architecture-specific verification that the stack pointer points where
it should.
>
> > +When you register the first probe at probepoint or unregister the
> > +last probe probe at a probepoint, Uprobes asks Utrace to "quiesce"
> > +the probed process so that Uprobes can insert or remove the breakpoint
> > +instruction. If the process is not already stopped, Utrace sends it
> > +a SIGSTOP. If the process is running an interruptible system call,
> > +this may cause the system call to finish early or fail with EINTR.
>
> Is this a limitation of utrace at the moment? Or will it be possible in
> the future for utrace to "quiesce" a process without a "user visible"
> SIGSTOP signal?
That's one for Roland.
>
> Cheers,
>
> Mark
Good ideas. Thanks.
Jim