Martin Cermak [Wed, 11 Nov 2015 09:23:16 +0000 (10:23 +0100)]
Fix synthetic probes generation for written and unread private globals.
Mangling a private global happens based on the name of the file where the
variable got defined. Stap automatically prints written and unread globals
and private globals at the end of the session. For autogeneration purposes
of respective derived synthetic probes, we need to keep track of the file
where given variable got defined.
Josh Stone [Wed, 11 Nov 2015 02:03:08 +0000 (18:03 -0800)]
Avoid undefined behavior in get_number_param
An uninitialized assignment led the compiler to assume it must be
initialized after all, which led to an optimization reversing the
conditions of an '&&'. This triggered valgrind when the condition
actually was uninitialized after all. See GCC PR68281.
Ditch the long and long long get_number_param for just int64_t, which is
just a pass-through to get_param. A few fields are now int64_t to
match, so they can never truncate input values on 32-bit systems. The
other Dwarf_Addr get_number_param is now careful to only copy good
values from get_param.
Just to be paranoid, all base_query and dwarf_query POD members are now
zeroed in the initializer lists, even when the constructor will set a
better value right away.
Martin Cermak [Tue, 10 Nov 2015 07:32:05 +0000 (08:32 +0100)]
Mark selected internal tapset global vars as '@__private30'.
Mark tapset global vars, that are only intended to be used
internally within given tapset, as '@__private30'. This
restriction appears to have no imparct from the testsuite
perspective. However, it's still possible that some user
script may rely on one of those. This restriction can be
worked around using 'stap --compatible=2.9' setting.
Martin Cermak [Sat, 7 Nov 2015 14:43:29 +0000 (09:43 -0500)]
Mark internal tapset functions as '@__private30' where possible.
Add '@__private30' macro that gives 'private' keyword for compatibility
mode 3.0 and higher. Mark internal tapset functions as private using
it where possible. Testcases examining private tapset functions must
now use the --compatible=2.9 option.
Martin Cermak [Thu, 5 Nov 2015 20:50:55 +0000 (15:50 -0500)]
Prefix global functions in the generated code.
Add '__global_' prefix to global functions in the generated code.
Treat selected internal synthetic functions as private.
Fix using a private function in user script and test-cover it.
* elaborate.cxx: Correctly match global function names to respective
functiondecls. Match synthetic private functions coming from
tapset-procfs.cxx (*_procfs_value_*), and tapsets.cxx
(_sdt_arg_get_addr_) to respective functiondecls.
* main.cxx: Update the --dump-functions feature to correctly deal with
new internal function names.
* parse.cxx: Add the "__global_" prefix to global functions in the
generated code.
* staptree.cxx: Update functiondecl::printsig() so that it outputs
unmangled function name.
* tapset-procfs.cxx: Treat internal synthetic functions
'*_procfs_value_get*', '*_procfs_value_set*', '*_procfs_value_append*'
as private.
* tapsets.cxx: Treat internal synthetic functions '*_sdt_arg_get_addr_*'
as private.
* optim_arridx.exp: Adapt the testcase to new naming.
* private.exp: Add subtest examining use of a private function in
user script.
Josh Stone [Tue, 3 Nov 2015 19:42:27 +0000 (11:42 -0800)]
Hide "__global_" mangling from the module parameters
This mangling is an implementation detail of the module, and we don't
want to introduce a version dependency for staprun and stapdyn which
understand this. Leave the parameter names unmangled and simply map
them to the mangled variable internally.
This also makes sure we don't expose the new private variables as
parameters at all, not even in mangled form.
Martin Cermak [Tue, 3 Nov 2015 15:16:36 +0000 (10:16 -0500)]
Make private functions play well with default optimizations.
Without this update, duplicate_function_remover::visit_functioncall()
would treat two different private functions of the same name as one
("Changing B reference to A reference"). This would happen with default
optimizations (without -u). This fix is analogical to b834d412e356
(mcermak/private branch) which does similar thing for private globals.
Martin Cermak [Tue, 3 Nov 2015 07:55:09 +0000 (02:55 -0500)]
Make test outut easier to compare among runs (PR19135).
Drop PIDs and auxiliary counters from output of listing_mode.exp
valid_pid.exp and bench.exp. With this update test outputs are easier
to compare between testsuite runs and still well debuggable.
David Smith [Mon, 2 Nov 2015 22:24:20 +0000 (16:24 -0600)]
Make a few small tweaks to modules_out_of_tree.exp.
* testsuite/systemtap.base/modules_out_of_tree.exp: A few tweaks, move all
the generated files to a new temporary directory and don't remove the
temporary directory if we're in verbose mode.
Martin Cermak [Mon, 2 Nov 2015 16:11:54 +0000 (17:11 +0100)]
Set input.check_compatible at the lexer ctor time.
Before this update input.check_compatible has been set after lexer
ctor finished. This was too late for has_version() to work properly
within the lexer ctor.
Martin Cermak [Fri, 30 Oct 2015 17:03:19 +0000 (18:03 +0100)]
Introduce "private" keyword for tapsets and user scripts.
With this update global variables and functions may be marked "private"
limiting their scope to a tapset or usersctipt file per PR19136.
* NEWS: Add entry.
* elaborate.cxx: Match names to decls within find_var(), update
generation of synthetic probes within add_global_var_display().
* parse.cxx: Recognize the "private" keyword.
* stapdyn/mutator.cxx: Pass global variable names to the dyninst
runtime with "__global_" prefix.
* staprun/staprun_funcs.c: Pass global variable names to the kernel
runtime with "__global_" prefix.
* tapset-utrace.cxx, tapsets.cxx: Prefix internal globals that are
bypassing the parser.
* util.cxx, util.h: Add detox_path() as a support function.
Josh Stone [Thu, 29 Oct 2015 00:43:23 +0000 (17:43 -0700)]
stap: buildrun's -Werror is not really optional
We really, really do want -Werror in kbuild invocations, else things
like "warning: passing argument from incompatible pointer type" will be
allowed, and that's almost certainly a fatal problem for kernel code.
This particular warning first shows up in stapconf checks for tracepoint
features, which then leads to uncompilable code elsewhere, so at least
it fails nicely. But we don't want *any* warnings to sneak through.
The string in buildrun.cxx is now slightly obfuscated so it may escape
Gentoo's blunt sed removal.
This also removes the undocumented, obfuscated, *evil* option from the
command which omitted -Werror. If you're really so sure that you don't
need it, you must now talk to this guy:
Frank Ch. Eigler [Tue, 27 Oct 2015 22:34:27 +0000 (18:34 -0400)]
PR14431 cont'd: syscalls testsuite fallout
jistone helpfully noted that the innocuous tapset changes broke the
syscall testsuite, since the latter looks for particular idiosyncratic
hex patterns. Lovingly adjusting them, a single line at a time, until
[7]f+ turns into the newly consistentified 0x[7]f+, etc.
Frank Ch. Eigler [Tue, 27 Oct 2015 22:33:01 +0000 (18:33 -0400)]
PR14431 cont'd: user_string_n2_quoted()
Turn this lower-level tapset function into a pure-script alternative
that uses the same new standard 0xHEX notation for failed accesses
that the {user,kernel}_string_quoted() ones do.
Frank Ch. Eigler [Tue, 27 Oct 2015 22:26:24 +0000 (18:26 -0400)]
testsuite: wrap /usr/bin/stress invocations in /usr/bin/timeout -k
On some platforms (I'm winking at you, rawhide!), stress(1) could get
into a hung state when running under stap probes, even though it was
given the "-t TIMEOUT" parameter. Now we wrap it in timeout(1), which
makes sure that stress won't last forever.
Josh Stone [Wed, 28 Oct 2015 00:42:32 +0000 (17:42 -0700)]
runtime: reject overlong UTF-8, output as raw octal instead
Since string_quoted is meant to unambiguously represent its output, it's a
bad idea to let different UTF-8 byte sequences lead to the same Unicode
code point. This can only happen if you allow 0-padded UTF-8 that ends up
longer than necessary. It's especially bad if you let long zero encodings
like 0xC0 0x80 be treated as a string terminator!
If any UTF-8 was decoded from more bytes than its value should have needed,
just write it verbatim with octal escapes instead.
David Smith [Tue, 27 Oct 2015 18:54:19 +0000 (13:54 -0500)]
Update the mmreclaim.stp example's "meta" file.
* testsuite/systemtap.examples/memory/mmreclaim.meta (test_support): Relax
test support requirements, only checking for any tracepoint that looks
like "mm_*reclaim*".
Josh Stone [Fri, 23 Oct 2015 19:22:22 +0000 (12:22 -0700)]
forktracker: add TIDs to the trace
It can be confusing for users to see that a PID forked into the same PID,
but really this is just a new thread in that process. Printing the TIDs
makes this more obvious.
David Smith [Fri, 23 Oct 2015 18:48:12 +0000 (13:48 -0500)]
Update the mmfilepage.stp and net_xmit_json.stp example scripts.
* testsuite/systemtap.examples/memory/mmfilepage.stp: Update script for
newer kernels (RHEL7+).
* testsuite/systemtap.examples/memory/mmfilepage.meta (test_support):
Relax support requirements.
* testsuite/systemtap.examples/network/net_xmit_json.stp: Make the
"net_dev_start_xmit" tracepoint optional, since it doesn't exist in some
kernels.
* testsuite/systemtap.examples/network/net_xmit_json.meta: Ditto.
* testsuite/systemtap.examples/profiling/fileline-profile.meta: Correct
typo so that the script gets built.
David Smith [Thu, 22 Oct 2015 21:00:30 +0000 (16:00 -0500)]
Update the 'autofs4' example script for current kernels.
* testsuite/systemtap.examples/network/autofs4.stp: Update for kernels >
RHEL5-era kernels.
* testsuite/systemtap.examples/network/autofs4.meta (test_support): Remove
'test_support' line since the script should work everywhere now.
Josh Stone [Thu, 22 Oct 2015 01:34:50 +0000 (18:34 -0700)]
runtime: handle the edges of utf-lengths better
The edge cases of UTF-8 lengths in _stp_convert_utf32 were using "<"
instead of "<=", causing some values to output overlong encodings.
(Some of these are noncharacters anyway, but they're still encodable.)
Frank Ch. Eigler [Wed, 21 Oct 2015 20:11:14 +0000 (16:11 -0400)]
/*stable*/ embedded-c functions: restore check for arity-0
During the various rewrites, this check got lost, and caused
arity-1 putatively-/*stable*/ functions to be recognized in
pass 2 but cause a SEGV in pass 3.
Frank Ch. Eigler [Wed, 21 Oct 2015 18:02:57 +0000 (14:02 -0400)]
PR14431: improve quoting of pretty-printed strings in $foo$$
Switch to using {kernel,user}_string_quoted() for expansion of
char-ish $var$ expansions. Required adding utf16/32 wrappers
for same, made easier by new tapset function string_quoted().
David Smith [Tue, 20 Oct 2015 21:25:36 +0000 (16:25 -0500)]
Update the fslatency-nd.stp and fsslower-nd.stp examples for newer kernels.
* testsuite/systemtap.examples/lwtools/fslatency-nd.stp: Updated for newer
kernels, without the kernel functions "do_sync_read", "do_sync_write",
and "sock_aio_write".
* testsuite/systemtap.examples/lwtools/fsslower-nd.stp: Ditto.
David Smith [Tue, 20 Oct 2015 18:19:42 +0000 (13:19 -0500)]
Fixed PR19120 by rearranging the list of modules to search in @cast().
* tapset/linux/dentry.stp (fullpath_struct_path): Rearrange the list of
modules to search for 'struct path' so that looking in "<linux/path.h>"
comes last, so that RHEL5-era kernels don't make it that far. That
include file doesn't exist on RHEL5-era kernels, which causes kbuild
warnings.
David Smith [Tue, 20 Oct 2015 17:45:59 +0000 (12:45 -0500)]
Update several of the example scripts for better kernel coverage.
* testsuite/systemtap.examples/network/tcp_trace.stp: Handle change in the
tcp_sock structure in newer kernels.
* testsuite/systemtap.examples/process/pfiles.stp: Add changes for
RHEL6-era kernels and kernels >= 4.0.
* testsuite/systemtap.examples/process/pstree.stp: Handle kernels < 3.7.
* testsuite/systemtap.examples/process/rlimit_nofile.stp: Add a fallback
for when the kernel function __alloc_fd() doesn't exist.
* testsuite/systemtap.examples/process/schedtimes.stp: Handle newer
kernels where the "sched_wakeup" tracepoint doesn't have a 'success'
argument.
* testsuite/systemtap.examples/profiling/fileline-profile.stp: Handle
32-bit kernels better by making the global array smaller.
* testsuite/systemtap.examples/profiling/pf3.stp: Ditto.
* testsuite/systemtap.examples/profiling/pf4.stp: Ditto.
* testsuite/systemtap.examples/io/slowvfs.meta: Add 'test_support' line.
* testsuite/systemtap.examples/process/procmod_watcher.meta: Ditto.
David Smith [Mon, 12 Oct 2015 16:17:00 +0000 (11:17 -0500)]
Fixed PR6525 by adding functions to get a task's cwd and exe path.
* tapset/linux/task.stp (task_cwd_path:long): New function.
(task_exe_file): Ditto.
* buildrun.cxx (compile_pass): Add export test for "get_mm_exe_file()".
* testsuite/systemtap.base/task_paths.exp: New test case.
* testsuite/systemtap.base/task_paths.stp: New test file.
* testsuite/buildok/task-embedded.stp: Added compile tests for new
functions.
Stan Cox [Fri, 9 Oct 2015 15:47:21 +0000 (11:47 -0400)]
Update systemtap.apps tests.
* (python.exp): Untest by noting that this test was replaced by
the example suite python test.
* (xulrunner.exp): Untest if gtk+-2.0 configure requirement fails
* (py?example.meta): Require python debuginfo
* (python.stpm): Update python3-lib version.
Remove the "-g" flag from both {k,u}probe tests; it's not needed, and
for the kernel case can trigger weird hangs due to overriding of the
blacklist. Relax the output matching patterns, perhaps due to
different ordering of task workqueues in 4.3.0-rc. Produce more
diagnostics upon failure.
David Smith [Thu, 8 Oct 2015 14:38:51 +0000 (09:38 -0500)]
Improve the nfsdtop.stp example script.
* testsuite/systemtap.examples/network/nfsdtop.stp: By default, now
returns 'dirname/filename' for the last files looked up for each
client. With any command line argument, returns the full pathname.
David Smith [Wed, 7 Oct 2015 19:01:20 +0000 (15:01 -0400)]
tapset: syscall.fadvise64_64.return
Tolerate occasionally mis-dwarf'd sys_fadvise64_64 function by
permitting the .return probe to resolve without a $return value.
Convert to if @defined()-based hack.
elaboration: decorate more pass-2 errors with probe derivation chain / function name
semantic_pass_types() carries out $context variable resolution, but
didn't identify what probe the $variable might have come from. This
is especially tricky if the derived_probe is many wildcarding/aliasing
levels away from the input probe point. We now print per-probe (or
analogous per-function) naming during the semantic_pass_types loops
via semantic_error chaining.
David Smith [Wed, 7 Oct 2015 15:14:15 +0000 (10:14 -0500)]
Fix PR19000 by making task_ns_pid() and task_ns_tid() more robust.
* runtime/linux/namespaces.h (from_target_pid_ns): Make sure the task
struct is valid before using it. The task_ns_pid() and task_ns_tid()
tapset functions call from_target_pid_ns().
(_stp_task_struct_valid): New function.
dsmith pointed out that switching to a quoted string is not always the
right thing to do, esp. from a backward compatibility point of view.
We now formalize a policy guideline in DEVGUIDE that *_string2 be used
when pointers should be valid but might be 0, which is the situation
with these probe points.
David Smith [Mon, 5 Oct 2015 18:10:49 +0000 (13:10 -0500)]
Fix PR19065 by making task_fd_lookup() more robust.
* tapset/linux/task.stp (task_fd_lookup): No longer write to the
task_struct memory. We check to make sure the memory is valid to read
from, but that doesn't mean it is valid to write to.
Martin Cermak [Mon, 5 Oct 2015 13:58:36 +0000 (15:58 +0200)]
Fix compile-time issues for a few _struct_compat_*_u() functions.
* tapset/linux/aux_syscalls.stp: Fix _struct_compat_itimerspec_u(),
and _struct_itimerval_u() so that they compile on non CONFIG_COMPAT
systems. Fix _struct_sched_attr_u() so that it compiles on kernels
not having struct sched_attr, i.e. when kernel_v < "3.14".
* testsuite/buildok/aux_syscalls-embedded.stp: Remove unneeded
constraint for _struct_sched_attr_u() testing.
David Smith [Fri, 2 Oct 2015 16:55:19 +0000 (11:55 -0500)]
Fix PR17920 by adding task_fd_lookup().
* tapset/linux/task.stp (task_fd_lookup): New function.
* tapset/linux/dentry.stp (fullpath_struct_file): Ditto.
* testsuite/buildok/dentry-embedded.stp: Added compile test for
fullpath_struct_file().
* testsuite/buildok/task-embedded.stp: Added compile test for
task_fd_lookup().
* testsuite/systemtap.base/task_fd_lookup.exp: New testcase for
task_fd_lookup.
* testsuite/systemtap.base/task_fd_lookup.stp: Ditto.
* testsuite/systemtap.base/task_fd_lookup.c: Ditto.
David Smith [Fri, 2 Oct 2015 15:36:35 +0000 (10:36 -0500)]
Fix PR19021 by adding support for synthetic filesystems to task_dentry_path().
* tapset/linux/dentry.stp (task_dentry_path): Add better support for
synthetic filesystems, like pipes, sockets, etc.
(fullpath_struct_path): Add better RHEL5 support.
Martin Cermak [Fri, 2 Oct 2015 14:48:54 +0000 (16:48 +0200)]
Explicitly set the return type for __stack_raw() and __ustack_raw().
Missing return type defs may cause "unresolved type" error when
calling a function without enough context for type inference to work
like e.g. `stap -e 'probe oneshot{println(__stack_raw(0))}'`.
David Smith [Thu, 1 Oct 2015 21:03:08 +0000 (16:03 -0500)]
Fix PR19045 by updating _stp_text_str() (called by kernel_string_quoted()).
* runtime/stp_string.c (_stp_text_str): Use _stp_read_address() to read
kernel memory also, not just user memory. This should keep us safe from
invalid pointers.
David Smith [Thu, 1 Oct 2015 18:50:21 +0000 (13:50 -0500)]
Fix PR19043 by updating __bio_ino(), __rqstp_gid() and __rqstp_uid().
* tapset/linux/nfsd.stp (__rqstp_uid): Make sure all pointers are valid
before using them.
(__rqstp_gid): Ditto.
* tapset/linux/ioblock.stp (__bio_ino): Ditto.
(bio_rw_num): Rewrite using embedded-C.
Eliminating the timer.s() wait, putting in a fixed 'sleep NN'
into the -c CMD string makes stopping condition clearer and
overall CMD exit value fixed (0).
On a rawhide smp box, it was possible to see corrupted lines of text
from this test's multiple concurrent outputs, and thus lose counts.
The strings are revised, and exit timing managed mroe carefully,
to make the run more repeatable.
David Smith [Fri, 25 Sep 2015 17:25:41 +0000 (12:25 -0500)]
Set up kfails for testsuite/buildok/stap_staticmarkers-detailed.stp.
* testsuite/systemtap.pass1-4/buildok.exp: For
buildok/stap_staticmarkers-detailed.stp, kfail if we don't have uprobes
or if we're not running as root. Only root can read "staprun".