This is the mail archive of the
mailing list for the systemtap project.
Re: [PATCH] Linux Kernel Markers
- From: Vara Prasad <prasadav at us dot ibm dot com>
- To: "Martin J. Bligh" <mbligh at google dot com>
- Cc: Ingo Molnar <mingo at elte dot hu>, Mathieu Desnoyers <mathieu dot desnoyers at polymtl dot ca>, "Frank Ch. Eigler" <fche at redhat dot com>, Paul Mundt <lethal at linux-sh dot org>, linux-kernel <linux-kernel at vger dot kernel dot org>, Jes Sorensen <jes at sgi dot com>, Andrew Morton <akpm at osdl dot org>, Tom Zanussi <zanussi at us dot ibm dot com>, Richard J Moore <richardj_moore at uk dot ibm dot com>, Michel Dagenais <michel dot dagenais at polymtl dot ca>, Christoph Hellwig <hch at infradead dot org>, Greg Kroah-Hartman <gregkh at suse dot de>, Thomas Gleixner <tglx at linutronix dot de>, William Cohen <wcohen at redhat dot com>, ltt-dev at shafik dot org, systemtap at sources dot redhat dot com, Alan Cox <alan at lxorguk dot ukuu dot org dot uk>
- Date: Tue, 19 Sep 2006 09:06:48 -0700
- Subject: Re: [PATCH] Linux Kernel Markers
- References: <20060918234502.GA197@Krystal> <20060919081124.GA30394@elte.hu> <451008AC.firstname.lastname@example.org>
Martin J. Bligh wrote:
Ingo Molnar wrote:
* Mathieu Desnoyers <email@example.com> wrote:
+ prompt "MARK code marker behavior"
+ Change markers for a function call.
as indicated before in great detail, NACK on this profileration of
marker options, especially the function call one. I'd like to see
_one_ marker mechanism that distros could enable, preferably with
zero (or at most one NOP) in-code overhead. (You can of course patch
whatever extension ontop of it, in out-of-tree code, to gain further
performance advantage by generating direct system-calls.)
There might be a hodgepodge of methods and tools in userspace to do
debugging, but in the kernel we should get our act together and only
take _one_ (or none at all), and then spend all our efforts on
improving that primary method of debug instrumentation. As
kprobes/SystemTap has proven, it is possible to have zero-overhead
Furthermore, for such a patch to make sense in the upstream kernel,
downstream tracing code has to make actual use of that NOP-marker.
I.e. a necessary (but not sufficient) requirement for upstream
inclusion (in my view) would be for this mechanism to be used by LTT
and LKST. (again, you can patch LTT for your own purposes in your own
patchset if you think the performance overhead of probes is too much)
You know ... it strikes me that there's another way to do this, that's
zero overhead when not enabled, and gets rid of the inflexibility in
kprobes. It might not work well in all cases, but at least for simple
non-inlined functions, it'd seem to.
Why don't we just copy the whole damned function somewhere else, and
make an instrumented copy (as a kernel module)? Then reroute all the
function calls through it, instead of the original version. OK, it's
not completely trivial to do, but simpler than kprobes (probably
doing the switchover atomically is the hard part, but not impossible).
There's NO overhead when not using, and much lower than probes when
That way we can do whatever the hell we please with internal variables,
however GCC optimises it, can write flexible instrumenting code to just
about anything, program in C as God intended, etc, etc. No, it probably
won't fix every case under the sun, but hopefully most of them, and we
can still use kprobes/djprobes/bodilyprobes for the rest of the cases.
It is an interesting idea but there appears to be following hard issues
(some of which you have already listed) i am not able to see how we can
1) We are going to have a duplicate of the whole function which means
any significant changes in the original function needs to be done on the
copy as well, you think maintainers would like this double work idea.
2) Inline functions is often the place where we need a fast path to
overcome the current kprobes overhead.
3) As you said it is not trivial across all the platforms to do a switch
to the instrumented function from the original during the execution.
This problem is similar to the issue we are dealing with djprobes.