Felix Lu [Wed, 28 Oct 2015 18:44:35 +0000 (14:44 -0400)]
PR18848: add most functionality to curses interface
Currently, a file is used to redirect stp_print output to curses window.
Previous curses interface code in mainloop got refactored into monitor.c.
* runtime/monitor.c: New file. procfs functions for stp_print redirection
* runtime/print_flush.c: Redirect stp_print to curses interface in
monitor mode
* staprun/monitor.c: New file implementing curses interface
* staprun/mainloop.c: Move code into monitor.c
* staprun/Makefile.*: Link json-c and curses library, include monitor.c
to source list
* translate.cxx: Create procfs file for stp_print redirection
Felix Lu [Thu, 15 Oct 2015 18:51:24 +0000 (14:51 -0400)]
PR18848: new command line argument -M for staprun
Pass stap --monitor command to staprun.
* buildrun.cxx: add -M argument to flags
* staprun/common.c: -M option to parse_args
* staprun/mainloop.c: add a timeout to pselect in monitor mode
to prevent blocking and allow the procfs monitor interface to
be polled.
* staprun/staprun.h: monitor extern declaration
Felix Lu [Wed, 7 Oct 2015 15:33:59 +0000 (11:33 -0400)]
PR18848: use jiffies tapset function
Switch the implementation of uptime to use the jiffies and hz tapset
functions instead of using embedded C
* elaborate.cxx: remove old embedded C function to get uptime. Added
a synthetic begin probe to obtain starting time. Synthetic variables
are now hidden in monitor mode and not reset.
* translate.cxx: remove jiffies inclusion in monitor mode.
Felix Lu [Tue, 6 Oct 2015 20:15:07 +0000 (16:15 -0400)]
PR18848: new procfs monitor mode option
Presents a procfs read interface to obtain data about the running
script and another procfs write interface to control the script
* cmdline.cxx: new "monitor" option
* cmdline.h: new field in enum
* elaborate.cxx: new functions monitor_mode_read and monitor_mode_write
synthesizes procfs probes for the interfaces
* session.cxx: new monitor flag
* session.h: new monitor flag
* translate.cxx: extra code is generated to provide statistics for
monitor mode, function emission is delayed to allow stap_probes to
be first defined
Martin Cermak [Fri, 20 Nov 2015 11:22:05 +0000 (12:22 +0100)]
Update the PR19241 fix so that 'out_fd' gets correctly handled.
This is an update to commit d879410b. Before this update, stap
in "stream mode" was using invalid output file descriptor per
https://sourceware.org/bugzilla/show_bug.cgi?id=19241#c2.
Martin Cermak [Fri, 20 Nov 2015 10:00:30 +0000 (11:00 +0100)]
Allow buildok/{rpc-all-probes,rpc-detailed} to access private var.
Tests buildok/rpc-all-probes.stp and buildok/rpc-detailed.stp are
examining '@__private30 global __rpc_create_args' array. That can
only be accessed using '--compatible=2.9' by the test script.
This approach to testing private stuff will probably need to change
in the future. For now, this update aligns buildok/{rpc-all-probes,\
rpc-detailed}.stp to other buildok tests examining private variables
or functions.
William Cohen [Thu, 19 Nov 2015 22:05:12 +0000 (17:05 -0500)]
Avoid using the nomencl compatible option
Some versions of nomencl such as the one on RHEL 5 are missing the
compatible option. Thus, tutorial.tex should avoid using it so the
documentation can be built.
David Smith [Thu, 19 Nov 2015 18:38:09 +0000 (12:38 -0600)]
Improve kmodule.exp by adding tests that load the module after stap.
* testsuite/systemtap.base/kmodule.exp: Add subtests that see if the same
scripts work if the module is loaded after the systemtap module is
loaded. This currently fails for non-dwarf kprobe probes (PR19216).
David Smith [Wed, 18 Nov 2015 19:03:44 +0000 (13:03 -0600)]
Refix PR18889 by switching to symbol_name+offset module probing.
* tapsets.cxx: Add symbol_name+offset support for dwarf kprobes present in
modules. This is somewhat based on work by James Bottomley from the
following email:
<https://sourceware.org/ml/systemtap/2008-q3/msg00182.html>
* runtime/transport/symbols.c (_stp_module_notifier): On kernel's >= 3.16,
use symbol_name+offset probing for modules.
* runtime/linux/kprobes.c: Switch to using symbol_name+offset probing for
modules.
David Smith [Wed, 18 Nov 2015 18:53:33 +0000 (12:53 -0600)]
Revert "PR18889 part: module-init notification via module_{load,free} tracepoints"
This reverts commit 2278079efc01124dd509241f6c6eadbd6e19cb2a. This code
will be replaced with a new method for handling probes in kernel module's
init sections.
David Smith [Wed, 18 Nov 2015 18:52:08 +0000 (12:52 -0600)]
Revert "PR18889: switch to STP_TRACEPOINT* frontend for kernel tracepoint registration"
This reverts commit 2e67c14dad1c661d2ce0b0ed218b371c1af218ba. This code
will be replaced with a different method for handling probes in kernel
module's init sections.
Martin Cermak [Wed, 18 Nov 2015 10:00:07 +0000 (11:00 +0100)]
Make the "shebang trick" work for RPM packaged testcases.
Some testcases use the "shebang trick" described in stap_run_batch()
testsuite function. Commit d2995e6f breaks this for RPM packaged
testcases by putting the absolute path to stap into the shebang.
This update reverts cb69017 update to stap_run_batch() and fixes the
specfile shebang rewriter instead, so that it only applies to
systemtap examples contained in the systemtap-client package, where
the "shebang trick" is unwelcome. With this update, testcases
contained in the systemtap-testsuite RPM remain unchanged, so that
the "shebang trick" works for them.
Need to increase --rlimit-cpu, since especially certain buildok
tests seem to reach the old value easily. It turns out that not
only "32-bit arm machines are quite slow", so not distinguishing
the machine type any more and just raising the treshold:
The goal is to raise the limit for low cost. Note that the
"-E 'probe timer.s(900) ...'' limit affects the generated source
that some testcases rely on verbatimely. So setting --rlimit-cpu
just slightly under 15 mins should improve test results without
need to modify those testcases source.
The stap_run_batch() update is needed to deal properly with RPM
packaged testcases, which have slightly modified source code
compared to their git versions due to commit d2995e6f. This update
is an addition to commit 6b131ba7 making it work with RPM packaged
testcases too.
Martin Cermak [Sat, 14 Nov 2015 18:17:27 +0000 (19:17 +0100)]
Terminate parsing on duplicate variable or function declaration.
This fixes condition that makes the parser stop on attempt to
declare duplicate global variable or function. It also (newly)
makes the parser terminate on global versus private variable or
function declaration attempt.
This update makes parseko/eleven.stp and parseko/twelve.stp
expectedly fail. It also adds parseko/private0[12].stp that
expectedly fail on attempt do declare conflicting global versus
private variable or function.
Martin Cermak [Fri, 13 Nov 2015 15:21:15 +0000 (16:21 +0100)]
Fix PR19241 by scanning for available CPUs.
Per PR19241/rhbz1280321, staprun was failing to start when CPU0
was offline or offline. In bulk mode, staprun was failing if
any CPU but the last one was offline. This patch fixes it by
scanning for available online CPUS up to NR_CPUS.
Martin Cermak [Fri, 13 Nov 2015 13:10:36 +0000 (14:10 +0100)]
Update user_string_n2_quoted() so that semok/context-globals.stp is happy.
Rename the local 'str' variable within user_string_n2_quoted() to '__str'
so that it doesn't collide with global variable of the same name defined
in the semok/context-globals.stp testcase.
For the most part, c_str() is just used for "%s" in warnings, errors,
and debug messages. We can afford to pay the cost of to_string() there,
while allowing string_ref to share memory the rest of the time.
NB: when using `to_string().c_str()`, it's important not to let that
pointer outlive the string temporary! It's good only for the range of
its full expression; if you need it longer, bind the string to a local
reference first.
Martin Cermak [Wed, 11 Nov 2015 09:23:16 +0000 (10:23 +0100)]
Fix synthetic probes generation for written and unread private globals.
Mangling a private global happens based on the name of the file where the
variable got defined. Stap automatically prints written and unread globals
and private globals at the end of the session. For autogeneration purposes
of respective derived synthetic probes, we need to keep track of the file
where given variable got defined.
Josh Stone [Wed, 11 Nov 2015 02:03:08 +0000 (18:03 -0800)]
Avoid undefined behavior in get_number_param
An uninitialized assignment led the compiler to assume it must be
initialized after all, which led to an optimization reversing the
conditions of an '&&'. This triggered valgrind when the condition
actually was uninitialized after all. See GCC PR68281.
Ditch the long and long long get_number_param for just int64_t, which is
just a pass-through to get_param. A few fields are now int64_t to
match, so they can never truncate input values on 32-bit systems. The
other Dwarf_Addr get_number_param is now careful to only copy good
values from get_param.
Just to be paranoid, all base_query and dwarf_query POD members are now
zeroed in the initializer lists, even when the constructor will set a
better value right away.
Martin Cermak [Tue, 10 Nov 2015 07:32:05 +0000 (08:32 +0100)]
Mark selected internal tapset global vars as '@__private30'.
Mark tapset global vars, that are only intended to be used
internally within given tapset, as '@__private30'. This
restriction appears to have no imparct from the testsuite
perspective. However, it's still possible that some user
script may rely on one of those. This restriction can be
worked around using 'stap --compatible=2.9' setting.
Martin Cermak [Sat, 7 Nov 2015 14:43:29 +0000 (09:43 -0500)]
Mark internal tapset functions as '@__private30' where possible.
Add '@__private30' macro that gives 'private' keyword for compatibility
mode 3.0 and higher. Mark internal tapset functions as private using
it where possible. Testcases examining private tapset functions must
now use the --compatible=2.9 option.
Martin Cermak [Thu, 5 Nov 2015 20:50:55 +0000 (15:50 -0500)]
Prefix global functions in the generated code.
Add '__global_' prefix to global functions in the generated code.
Treat selected internal synthetic functions as private.
Fix using a private function in user script and test-cover it.
* elaborate.cxx: Correctly match global function names to respective
functiondecls. Match synthetic private functions coming from
tapset-procfs.cxx (*_procfs_value_*), and tapsets.cxx
(_sdt_arg_get_addr_) to respective functiondecls.
* main.cxx: Update the --dump-functions feature to correctly deal with
new internal function names.
* parse.cxx: Add the "__global_" prefix to global functions in the
generated code.
* staptree.cxx: Update functiondecl::printsig() so that it outputs
unmangled function name.
* tapset-procfs.cxx: Treat internal synthetic functions
'*_procfs_value_get*', '*_procfs_value_set*', '*_procfs_value_append*'
as private.
* tapsets.cxx: Treat internal synthetic functions '*_sdt_arg_get_addr_*'
as private.
* optim_arridx.exp: Adapt the testcase to new naming.
* private.exp: Add subtest examining use of a private function in
user script.
Josh Stone [Tue, 3 Nov 2015 19:42:27 +0000 (11:42 -0800)]
Hide "__global_" mangling from the module parameters
This mangling is an implementation detail of the module, and we don't
want to introduce a version dependency for staprun and stapdyn which
understand this. Leave the parameter names unmangled and simply map
them to the mangled variable internally.
This also makes sure we don't expose the new private variables as
parameters at all, not even in mangled form.
Martin Cermak [Tue, 3 Nov 2015 15:16:36 +0000 (10:16 -0500)]
Make private functions play well with default optimizations.
Without this update, duplicate_function_remover::visit_functioncall()
would treat two different private functions of the same name as one
("Changing B reference to A reference"). This would happen with default
optimizations (without -u). This fix is analogical to b834d412e356
(mcermak/private branch) which does similar thing for private globals.
Martin Cermak [Tue, 3 Nov 2015 07:55:09 +0000 (02:55 -0500)]
Make test outut easier to compare among runs (PR19135).
Drop PIDs and auxiliary counters from output of listing_mode.exp
valid_pid.exp and bench.exp. With this update test outputs are easier
to compare between testsuite runs and still well debuggable.
David Smith [Mon, 2 Nov 2015 22:24:20 +0000 (16:24 -0600)]
Make a few small tweaks to modules_out_of_tree.exp.
* testsuite/systemtap.base/modules_out_of_tree.exp: A few tweaks, move all
the generated files to a new temporary directory and don't remove the
temporary directory if we're in verbose mode.
Martin Cermak [Mon, 2 Nov 2015 16:11:54 +0000 (17:11 +0100)]
Set input.check_compatible at the lexer ctor time.
Before this update input.check_compatible has been set after lexer
ctor finished. This was too late for has_version() to work properly
within the lexer ctor.
Martin Cermak [Fri, 30 Oct 2015 17:03:19 +0000 (18:03 +0100)]
Introduce "private" keyword for tapsets and user scripts.
With this update global variables and functions may be marked "private"
limiting their scope to a tapset or usersctipt file per PR19136.
* NEWS: Add entry.
* elaborate.cxx: Match names to decls within find_var(), update
generation of synthetic probes within add_global_var_display().
* parse.cxx: Recognize the "private" keyword.
* stapdyn/mutator.cxx: Pass global variable names to the dyninst
runtime with "__global_" prefix.
* staprun/staprun_funcs.c: Pass global variable names to the kernel
runtime with "__global_" prefix.
* tapset-utrace.cxx, tapsets.cxx: Prefix internal globals that are
bypassing the parser.
* util.cxx, util.h: Add detox_path() as a support function.
Josh Stone [Thu, 29 Oct 2015 00:43:23 +0000 (17:43 -0700)]
stap: buildrun's -Werror is not really optional
We really, really do want -Werror in kbuild invocations, else things
like "warning: passing argument from incompatible pointer type" will be
allowed, and that's almost certainly a fatal problem for kernel code.
This particular warning first shows up in stapconf checks for tracepoint
features, which then leads to uncompilable code elsewhere, so at least
it fails nicely. But we don't want *any* warnings to sneak through.
The string in buildrun.cxx is now slightly obfuscated so it may escape
Gentoo's blunt sed removal.
This also removes the undocumented, obfuscated, *evil* option from the
command which omitted -Werror. If you're really so sure that you don't
need it, you must now talk to this guy:
Frank Ch. Eigler [Tue, 27 Oct 2015 22:34:27 +0000 (18:34 -0400)]
PR14431 cont'd: syscalls testsuite fallout
jistone helpfully noted that the innocuous tapset changes broke the
syscall testsuite, since the latter looks for particular idiosyncratic
hex patterns. Lovingly adjusting them, a single line at a time, until
[7]f+ turns into the newly consistentified 0x[7]f+, etc.
Frank Ch. Eigler [Tue, 27 Oct 2015 22:33:01 +0000 (18:33 -0400)]
PR14431 cont'd: user_string_n2_quoted()
Turn this lower-level tapset function into a pure-script alternative
that uses the same new standard 0xHEX notation for failed accesses
that the {user,kernel}_string_quoted() ones do.
Frank Ch. Eigler [Tue, 27 Oct 2015 22:26:24 +0000 (18:26 -0400)]
testsuite: wrap /usr/bin/stress invocations in /usr/bin/timeout -k
On some platforms (I'm winking at you, rawhide!), stress(1) could get
into a hung state when running under stap probes, even though it was
given the "-t TIMEOUT" parameter. Now we wrap it in timeout(1), which
makes sure that stress won't last forever.
Josh Stone [Wed, 28 Oct 2015 00:42:32 +0000 (17:42 -0700)]
runtime: reject overlong UTF-8, output as raw octal instead
Since string_quoted is meant to unambiguously represent its output, it's a
bad idea to let different UTF-8 byte sequences lead to the same Unicode
code point. This can only happen if you allow 0-padded UTF-8 that ends up
longer than necessary. It's especially bad if you let long zero encodings
like 0xC0 0x80 be treated as a string terminator!
If any UTF-8 was decoded from more bytes than its value should have needed,
just write it verbatim with octal escapes instead.
David Smith [Tue, 27 Oct 2015 18:54:19 +0000 (13:54 -0500)]
Update the mmreclaim.stp example's "meta" file.
* testsuite/systemtap.examples/memory/mmreclaim.meta (test_support): Relax
test support requirements, only checking for any tracepoint that looks
like "mm_*reclaim*".
Josh Stone [Fri, 23 Oct 2015 19:22:22 +0000 (12:22 -0700)]
forktracker: add TIDs to the trace
It can be confusing for users to see that a PID forked into the same PID,
but really this is just a new thread in that process. Printing the TIDs
makes this more obvious.
David Smith [Fri, 23 Oct 2015 18:48:12 +0000 (13:48 -0500)]
Update the mmfilepage.stp and net_xmit_json.stp example scripts.
* testsuite/systemtap.examples/memory/mmfilepage.stp: Update script for
newer kernels (RHEL7+).
* testsuite/systemtap.examples/memory/mmfilepage.meta (test_support):
Relax support requirements.
* testsuite/systemtap.examples/network/net_xmit_json.stp: Make the
"net_dev_start_xmit" tracepoint optional, since it doesn't exist in some
kernels.
* testsuite/systemtap.examples/network/net_xmit_json.meta: Ditto.
* testsuite/systemtap.examples/profiling/fileline-profile.meta: Correct
typo so that the script gets built.
David Smith [Thu, 22 Oct 2015 21:00:30 +0000 (16:00 -0500)]
Update the 'autofs4' example script for current kernels.
* testsuite/systemtap.examples/network/autofs4.stp: Update for kernels >
RHEL5-era kernels.
* testsuite/systemtap.examples/network/autofs4.meta (test_support): Remove
'test_support' line since the script should work everywhere now.
Josh Stone [Thu, 22 Oct 2015 01:34:50 +0000 (18:34 -0700)]
runtime: handle the edges of utf-lengths better
The edge cases of UTF-8 lengths in _stp_convert_utf32 were using "<"
instead of "<=", causing some values to output overlong encodings.
(Some of these are noncharacters anyway, but they're still encodable.)
Frank Ch. Eigler [Wed, 21 Oct 2015 20:11:14 +0000 (16:11 -0400)]
/*stable*/ embedded-c functions: restore check for arity-0
During the various rewrites, this check got lost, and caused
arity-1 putatively-/*stable*/ functions to be recognized in
pass 2 but cause a SEGV in pass 3.
Frank Ch. Eigler [Wed, 21 Oct 2015 18:02:57 +0000 (14:02 -0400)]
PR14431: improve quoting of pretty-printed strings in $foo$$
Switch to using {kernel,user}_string_quoted() for expansion of
char-ish $var$ expansions. Required adding utf16/32 wrappers
for same, made easier by new tapset function string_quoted().
David Smith [Tue, 20 Oct 2015 21:25:36 +0000 (16:25 -0500)]
Update the fslatency-nd.stp and fsslower-nd.stp examples for newer kernels.
* testsuite/systemtap.examples/lwtools/fslatency-nd.stp: Updated for newer
kernels, without the kernel functions "do_sync_read", "do_sync_write",
and "sock_aio_write".
* testsuite/systemtap.examples/lwtools/fsslower-nd.stp: Ditto.
David Smith [Tue, 20 Oct 2015 18:19:42 +0000 (13:19 -0500)]
Fixed PR19120 by rearranging the list of modules to search in @cast().
* tapset/linux/dentry.stp (fullpath_struct_path): Rearrange the list of
modules to search for 'struct path' so that looking in "<linux/path.h>"
comes last, so that RHEL5-era kernels don't make it that far. That
include file doesn't exist on RHEL5-era kernels, which causes kbuild
warnings.
David Smith [Tue, 20 Oct 2015 17:45:59 +0000 (12:45 -0500)]
Update several of the example scripts for better kernel coverage.
* testsuite/systemtap.examples/network/tcp_trace.stp: Handle change in the
tcp_sock structure in newer kernels.
* testsuite/systemtap.examples/process/pfiles.stp: Add changes for
RHEL6-era kernels and kernels >= 4.0.
* testsuite/systemtap.examples/process/pstree.stp: Handle kernels < 3.7.
* testsuite/systemtap.examples/process/rlimit_nofile.stp: Add a fallback
for when the kernel function __alloc_fd() doesn't exist.
* testsuite/systemtap.examples/process/schedtimes.stp: Handle newer
kernels where the "sched_wakeup" tracepoint doesn't have a 'success'
argument.
* testsuite/systemtap.examples/profiling/fileline-profile.stp: Handle
32-bit kernels better by making the global array smaller.
* testsuite/systemtap.examples/profiling/pf3.stp: Ditto.
* testsuite/systemtap.examples/profiling/pf4.stp: Ditto.
* testsuite/systemtap.examples/io/slowvfs.meta: Add 'test_support' line.
* testsuite/systemtap.examples/process/procmod_watcher.meta: Ditto.
David Smith [Mon, 12 Oct 2015 16:17:00 +0000 (11:17 -0500)]
Fixed PR6525 by adding functions to get a task's cwd and exe path.
* tapset/linux/task.stp (task_cwd_path:long): New function.
(task_exe_file): Ditto.
* buildrun.cxx (compile_pass): Add export test for "get_mm_exe_file()".
* testsuite/systemtap.base/task_paths.exp: New test case.
* testsuite/systemtap.base/task_paths.stp: New test file.
* testsuite/buildok/task-embedded.stp: Added compile tests for new
functions.
Stan Cox [Fri, 9 Oct 2015 15:47:21 +0000 (11:47 -0400)]
Update systemtap.apps tests.
* (python.exp): Untest by noting that this test was replaced by
the example suite python test.
* (xulrunner.exp): Untest if gtk+-2.0 configure requirement fails
* (py?example.meta): Require python debuginfo
* (python.stpm): Update python3-lib version.
Remove the "-g" flag from both {k,u}probe tests; it's not needed, and
for the kernel case can trigger weird hangs due to overriding of the
blacklist. Relax the output matching patterns, perhaps due to
different ordering of task workqueues in 4.3.0-rc. Produce more
diagnostics upon failure.