This is the mail archive of the
mailing list for the systemtap project.
Re: breakpoint assistance (uprobes)
- From: Jim Keniston <jkenisto at us dot ibm dot com>
- To: Roland McGrath <roland at redhat dot com>
- Cc: systemtap at sources dot redhat dot com
- Date: Thu, 15 Mar 2007 15:59:52 -0700
- Subject: Re: breakpoint assistance (uprobes)
- References: <20070302212446.4D830180065@magilla.sf.frob.com>
On Fri, 2007-03-02 at 13:24 -0800, Roland McGrath wrote:
> I'm sorry for my long absence from the uprobes discussion. Rather
> than trying to respond to all the individual questions and proposals,
> I'll just start in on the design topics that I think are most useful
> to pursue first.
Thanks for kicking off this discussion, Roland. Unfortunately, I think
your email reached the systemtap list only yesterday.
> The central component of the currently going uprobes ideas that needs
> to be worked out is what I'll call breakpoint assistance. This is
> broadly useful for many utrace-based facilities to integrate, and any
> worthwhile implementation will become a modular component used by
> other things unrelated to systemtap. Today I'll discuss the major
> design choices in this component.
> On the status of utrace: it's now included in Andrew Morton's -mm tree;
> he tracks my current patchset from http://redhat.com/~roland/utrace/
> regularly. The best way anyone can accelerate the process towards
> full upstream acceptance is to participate in the public review and
> discussion. The correct forum for that is lkml (and CC: email@example.com),
> not this mailing list.
> Standard breakpoint insertion is straightforward and that is adequate.
> The complexity is in stepping over a breakpointed instruction to
> resume. There are three essential implementation strategies available
> (leaving aside pure emulation, and I don't know of a fifth).
> Stop all threads, remove breakpoint, single-step, re-insert, resume all.
> This is the traditional method used by debuggers. It is reliable and
> machine-independent, but has always been slow. Doing this via utrace in
> the kernel directly will be quite a bit faster than a user-level
> debugger can do. This is the natural fallback position if other methods
> don't pan out. The implementation of this is fairly straightfoward, but
> most of it will fall out of any new higher-level general facilities on
> top of utrace. It's not especially interesting to do it independently
> in support of systemtap/uprobes.
Uprobes folks have talked a fair amount amongst ourselves about
single-stepping in multithreaded apps. This approach was on our list,
but we haven't tried it because (1) it didn't seem very promising
performance-wise, and (2) it could really drive up the number of
SIGSTOPs utrace injects into the probed process.
> Single-step out of line, in the style of kprobes. I had been skeptical
> about doing this for user mode, but lately I'm feeling more optimistic.
> This appears to be the most promising direction in the short term.
> Its implementation issues are complex. I'll get into those in a
> separate thread momentarily.
Yeah, that's the approach we've bought into.
> Fancy VM tricks. This is the shangri-la path and the general subject
> has long been on my mind as future work for the utrace world. The
> general capability required to do this is useful for a variety of other
> things, including thread-specific breakpoint insertion and a better way
> to do breakpoint insertion generally, and VM watchpoints. I am full of
> ideas about this, but figuring it out requires some hard brainstorming
> with someone who knows the Linux VM code in much more detail than I do.
> I haven't had the chance to do that yet. (What I have in mind also
> needs to wait for a little bit of fancier utrace infrastructure
> development, too.) I don't know enough yet to be confident on the details
> of a design, but I know enough to be pretty sure that there is no simple
> "surgical strike" implementation plan that will work with acceptable
> properties for a production system. I certainly think we'll pursue this
> in the long term regardless, but I think it's too big a nut to crack soon.
This is not currently on our radar, at least as far as code probepoints