This is the mail archive of the
mailing list for the systemtap project.
Re: Systemtap vs Dtrace web page corrections.
- From: fche at redhat dot com (Frank Ch. Eigler)
- To: "James Dickens" <jamesd dot wi at gmail dot com>
- Cc: <systemtap at sources dot redhat dot com>
- Date: 13 Sep 2006 09:43:36 -0400
- Subject: Re: Systemtap vs Dtrace web page corrections.
- References: <email@example.com>
James Dickens wrote:
> Kernel Lock in:
> [...] Systemtap isn't even stable enough to run a complex script written 6
> months ago
That claim is at odds with a number of scripts in the test suite that
have survived unmodified for over a year. Perhaps you would care to
> So on this line, it would be that Systemtap has the greater amount
> of lock in, not just in the kernel but Systemtap it self and
> possibly even the tools used to compile the script.
You misunderstand the reference to 'lock step" or "lock in". The
issue is the extent to which systemtap requires cooperating patches in
the linux kernel.
Many of your other paragraphs similarly mix or confuse concepts such
as maturity, utility, capability. The purpose of the wiki page is
specifically to pry apart the issues.
> [...] Your scripts are basically C code modified to make it
> compatible with aC compiler. If one turns on guru mode, you are
> writing 100% C code, no way to consider it anything but C.
You misunderstand the scripting language and its implementation.
dtrace parses its language and generates byte codes; systemtap parses
its, and generates C code. They are both full-fledged language
On the execution side, dtrace has a bytecode interpreter, while
systemtap uses native code so no interpreter is needed, but that is
not relevant to the nature of source language.
> Control structures:
> Systemtap has full control structures as stated, but if a bug happens
> in the Systemtap script it can cause the box to crash.
> DTrace: doesn't have functions or loops per se, but you can work
> around this with a little thought. [...]
You are mixing things up. Systemtap control structures have nothing
to do with crashing boxes. Primitives like conditionals, loops,
recursion are translated to C code that includes protection against
accidents like infinite loops or recursion. The "halting problem" is
a red herring. I am aware of no bug related to control structure
capability appearing in many months.
> Variable Typing:
> [...] Because by implicitly setting the type of the variable you are
> now locked into that type, if the code you are probing changes a
> variable type your script no longer works. [...]
An example would help me understand the kind of change you envision as
posing a problem, and how dtrace would deal with the same issue.
> When Systemtap gets userland probes its inferred variable typing will
> become even more of a hindrance, lots of programs don't ship with
> debugging code embedded, so if you don't have the source code to
> recompile with debugging information Systemtap is useless, with DTrace
> it will try and make guesses at the data structure and include files
> and allow the user to type cast variables as needed, Systemtap does
> not have the native ability to process include files or handling data
> of unknown types.
It would be easier to answer your writings if they were broken into
sentences with individual propositions. Many of your ", so" splices
don't actually logically follow..
> Complex Reports:
> If Systemtap's report generating ability is so great why is there work
> done on a dashboard that is designed to make the reports look better?
Perhaps it's for the same reason that dtrace benefits from a graphical
> What limitations are you seeing in DTrace's report generating capability?
One appears to need post-processing perl scripts to generate anything
complex, like for example if data from more than one array needs to be
> Speculative tracing:
> Systemtap: you can't just wish this requirement a way, you can't judge
> whether or not you need a piece of data, in a complex system when the
> first event occurs, you have to store it until possibly many other
> events have occurred [...]
In the systemtap script language, one can implement the gist of this
from first principles: store that piece of data in global variables,
then emit actual trace messages later when the data is known to be
relevant. One variant of this is an exercise in the tutorial.
> Number of probe points:
> System tap, has too much bloat to have a truly unlimited number of
> probes [...] 1 million active probes requires 64MB of
> kernel space allocated [...]
You conflate the number of potential vs. actually activated probes.
> What is the highest number of Systemtap probes ever activated in an
> active script with out the machine falling over? In my tests of
> DTrace it is over 500,000. [...]
This is not sufficiently constrained to be answered. One can place
millions of probes that are never hit - or one probe that is hit
millions of times per second - or somewhere in between.
> Of course a good question is, what kind of kernel coder can't debug
> a problem when he knows what functions are being called, and by what
> function, how often and how much time its taking, and what functions
> it calls and complete userland and kernel stack tracing. Seems to be
> a pretty silly to do all that extra work and risk system stability
> in the name of probing every line in the kernel.
Practicing software engineers tend sometimes to put debugging code
such as printfs into places other than the beginning or ends of
functions. Talk to one to find out why.
> [...] Can Systemtap still probe arbitrary points in code if you
> don't have a binary with debugging information intact?
The granularity will likely be smaller - something closer to dtrace.
> Concurrent probes on multiprocessors:
> You can even run multiple copies of the same script with out
> problems, last time I tried this simple test on Systemtap, it
While this has nothing to do with multiprocessors, I am aware of no
such problem with systemtap. Care to share your script?
> [...] DTrace also understands the C struct construct so it can also
> read data stored in structs and use pointers stored in structs to
> access other data even if the program isn't compiled with debugging
> information. Systemtap requires debugging information to access data
> in structures even if the user knows the layout of the data
> structures. [...]
Without debugging information, this would require access to and
parsing of exactly matching header files from the source code. Are
you aware of the complications involved in this - and the likelihood
of producing garbage data (albeit safely) for mismatches? In any
case, nothing precludes future inclusion of this sort of function.