This is the mail archive of the
mailing list for the systemtap project.
Re: User-space probes: Plan B+
Frank Ch. Eigler wrote:
[...] Our interpretation of feedback from LKML and other sources was
1. Instrumentation can be coded entirely as a user-space app...I am not aware of this being a *requirement*. To my mind, it is an
unfortunate implementation artifact. Running in kernel space but in
the traced processes' user context would be better.
that uprobes would not be accepted without a reasonably friendly
That may be, but we would not be obligated to use such a thing. To
what extent can this work be layered on top of the kernel-side utrace
I agree we are not obliged to use such a thing in SystemTap and i am not
expecting we will use either but as the provider of an API in the kernel
we have to consider community feedback as we have to get their buy in to
get this accepted into the main line kernel. Jim, is trying to provide a
flexible interface so that it meets the community need but it gives the
freedom for us to just use the kernel mode probe handler which we can
dynamically load like we do it today.
I am not sure i followed exactly what you mean by searched in situ. What
i am thinking we will do for back trace is, probe handlers running in
the kernel will use platform specific kernel backtrace feature and dump
frame pointers out to userland in the order and then stap in the user
space consults the dwarf information and prints out the full stack along
with symbol information. BTB the stack backtrace i am imagining could be
full stack all the way from kernel to app, or just kernel stack. I
thought this method would work for Java as well as we need to do Java
stack traces in the userland anyway (as far as i know). Does this seem
reasonable or i am making it complicated than needed.
A few things, such as stack backtraces with symbol info, cannot be
accomplished entirely in the kernel, so you need user-mode
cooperation at some point anyway.
That's sort of a separate issue. We're not contemplating running live
elfutils-style dwarf searches during a probe handler to evaluate
backtraces. It is more likely that information required for a precise
backtrace would be encoded into the probing module in the form of an
extract of the dwarf unwind tables, and searched in situ.
Kernel-mode handlers can be registered in a kernel module. Registration
[...] 3. A user-mode tracer can invoke a previously registeredHow is this registration aspect supposed to work?
kernel-mode handler, so we have simple and efficient communication
between user- and kernel-mode instrumentation. [...]
requires the handler address, a unique name, and permission info (e.g.,
only root can call it, or only a process that has permission to run
uprobe_register() on the traced process can call it).
Handlers that we expect to have widespread use (e.g., collect
stack-trace addresses, report register contents, report memory
contents) could even be registered by the kernel.
FWIW, such designated "widespread-use" handlers are not in good match
with systemtap's preference for programmable handlers. It may be
helpful for specialized purposes such as LKET (say, by some sort of
clever handler body matching or new syntax), but not for general
I agree with you it is not easy (at least i can't think of one easily)
have a predefined parametrized handler with complicated filters and
aggregation which most of the systemtap ones are. That said, there are
simple yet common cases like back trace , register dump etc that many
people may like to do as pre-canned handler that you can activate, this
was one of the feedback items we got during the review.
The way i am thinking of how systemtap uses, you are very correct, user
space tracer is is a content free stub, i am thinking we can add that
to our existing stap, the real meat will be dynamically generated like
it is today. There may be others (which i doubt very much) who may want
to do it differently where they may write the handlers that run
completely in the user space and of course pay huge performance
overhead but they feel secure and safe. There may be others who only
want to do few little things like stack backtrace (this i see a very
much a possibility based on a customer requirement i got) using
predefined handlers. As you see Jim is trying to address many reqt.
hence forth API may be more complicated than needed.
Is somethingNo. I envision SystemTap generating both the user-space tracer app
keeping us from compiling & registering this instrumentation
dynamically, the same way we do normal kprobes handlers?
and the module that registers any needed ad hoc kernel handlers (as
well as the usual kernel-probing stuff).
Would this approach (compiling user-space probe handlers into
kernel-space registered but custom-built utrace handlers) make the
user-space tracer almost a content-free stub?
My guess is ( I am not betting the farm yet) most practical purposes
handlers copy the required information from the user space using
safecopy routines and run the handlers in the kernel.
Jim feel free to correct me if i am wrong.