Allow our /usr/bin/dtrace-impersonating usdt-.d file parser to take in
those .d files that use the "arguments passed to probe consumers"
extended syntax:
Frank Ch. Eigler [Fri, 20 Jan 2017 03:04:48 +0000 (22:04 -0500)]
runtime/unwind: don't warn about self-unwinding
If a kernel backtrace traversed the stap module itself, until this
patch, the runtime emitted a pair of _stp_warn()'s complaining about
not-quite-perfect unwind data. This is noisy and unhelpful, since a
user can't do anything about it, so we switch over to a dbug(1) type
message.
David Smith [Fri, 20 Jan 2017 19:06:40 +0000 (13:06 -0600)]
Handle F24 dropping ppc64 32-bit exe support in the testsuite.
* testsuite/lib/compile_flags.exp (arch_compile_init): New
function. Fedora 24 ppc64 has dropped 32-bit exe support. Now on ppc64
systems, we test the 32-bit support. If it doesn't work (or isn't
present), assume only 64-bit exe support.
(arch_compile_flags): For ppc64, return the number of compile flags
arch_compile_init() determined for this system.
(arch_compile_flag): Ditto.
* testsuite/lib/systemtap.exp (setup_systemtap_environment): Call
arch_compile_init().
Frank Ch. Eigler [Mon, 16 Jan 2017 18:23:57 +0000 (13:23 -0500)]
tweak python helper installation paths
Since our python helper includes C shared libraries, all the files
(including .egg-info) will get put under %python_sitearch
(/usr/lib64/...) rather than %python_sitelib (/usr/lib/...).
The translate-time stap-resolve-module-function* file belongs in
the -devel subrpm, not -runtime.
David Smith [Mon, 16 Jan 2017 15:29:38 +0000 (09:29 -0600)]
Fix the python helper module compiliation.
* python/Makefile.am: Fix a problem where we were using the system's copy
of sdt.h, instead of the source tree's copy of sdt.h.
* python/Makefile.in: Regenerated.
Frank Ch. Eigler [Mon, 16 Jan 2017 15:24:45 +0000 (10:24 -0500)]
testsuite infrastructure: change stap_run api
Formerly, the stap_run tcl proc accepted a target stap script file
name both as its first argument, as well as in the general stap
argument list at the end. It passed the former name as an extra file
name iff it existed in the $srcdir/$testdir. Some tests happened to
use test names that matched files, so triggered this stap_run
heuristic, even if the .stp files were specified some other way. This
led to extra stap arguments, leading to "unused $0/@0 ..." warnings,
leading to test failures.
Now, stap_run never does anything clever with its first parameter - it
just uses it to construct the dejagnu test case id/name. The .stp
file name (if any - if not using -e SCRIPT for example) is now
expected to be included by the testcase.exp as arguments at the end.
Frank Ch. Eigler [Mon, 16 Jan 2017 15:23:23 +0000 (10:23 -0500)]
tweak unused $n/@n warning for argv tapset users
Use the demangled (original) names in s.globals[] rather than mangled
name to heuristically identify use of the argv tapset (and thus
suppression of the warnings).
David Smith [Thu, 12 Jan 2017 20:31:29 +0000 (14:31 -0600)]
Add support for python 2 unicode objects.
* tapset/python2.stp: Add support for the unicode type. Move the python
module list into a macro, so we've only got one copy.
* testsuite/systemtap.python/python2.exp: Expect working unicode strings.
Tetsuo Handa [Wed, 11 Jan 2017 02:38:15 +0000 (21:38 -0500)]
PR21020: allow --compatible=3.0 java probe string $args without leaks
With some extra work, one can track which individual java parameters
were stringified & need free()ing, and which don't. That way,
--compatible=3.0 java probes can take string arguments again,
leaklessly.
Frank Ch. Eigler [Tue, 10 Jan 2017 20:16:58 +0000 (15:16 -0500)]
testsuite: suppress warnings from empty strings passed to stap
A test case passed a ""-valued tcl variable to a stap command line,
which due to tcl quoting rules means passing an empty string (rather
than nothing). Use some harmless real option for the empty case,
since we don't want the "" passed through to stap.
David Smith [Tue, 10 Jan 2017 19:21:22 +0000 (13:21 -0600)]
Swap python marker arguments around.
* python/HelperSDT/_HelperSDT.c (trace_callback): Swap the argument order
around so that the module name and key are always first.
* tapset-python.cxx: Expect the new marker argument order.
David Smith [Tue, 10 Jan 2017 18:35:25 +0000 (12:35 -0600)]
Handle python3 class variable references, like '$class->member'.
* tapset/python3.stp: Add support for most python objects (such as
classes), which use the same internal function to get their attributes.
* python/HelperSDT/_HelperSDT.c (PyInit__HelperSDT): For python version 3,
add an 'Init' marker.
* tapset-python.cxx (build): Add code to hook up to the new 'Init' marker.
* tapset/python2.stp: Added small simplification.
* testsuite/systemtap.python/python3.exp: Increase timeout values and
update.
PR21020: reorganize data passing abi for java method parameters
Under a new ABI, all java object types are supported for passing to
java-method probes, by converting them all within the JVM to strings.
The systemtap script side now gets $argX being all utf-char-pointers,
which are loaded into script-side string variables with synthetic
user_string_warn() function calls.
A backward compatibility ABI is preserved in the java helper class
as well as the translator, under --compatible=3.0.
The java testsuite is drastically simplified (and execution time
shortened) to one class. These now test string and object passing
too, with it and stapbm extended to be able to locate the HelperSDT
jar / .so files within the systemtap installation $prefix. This way,
the system /usr/lib/jvm/.... path does not need to be populated just
for stap tests to run. (We should sometime extend the tests to cover
all the argument arities, but eyeballing looks OK for now.)
elaborate.cxx: optimizations should elide elision warnings for synthetic probes
We need to check not only whether a token is associated with a
user-vs-tapset file, but also whether the file is entirely synthetic.
We don't want to warn users about the latter. Relevant to the
new java method probes' argXXX synthetic script variables.
The implementation of the system() tapset function involves sending a
ctl message via the runtime transport system, which can impose smaller
string length limits (STP_CTL_BUFFER_SIZE) than the usual systemtap
strings (MAXSTRINGLEN). Conditions like that resulted in a kernel
message (printk) being printed, which is a quiet failure.
New code increases the ctl-buffer-size from 256 to 384, and tweaks
_stp_ctl_send() to send an _stp_warn() in case of an overlength input.
The system() tapset function docs point out this possibility.
Frank Ch. Eigler [Mon, 26 Dec 2016 16:50:20 +0000 (11:50 -0500)]
staprun: stop forcing off kprobes-optimization for newer kernels
Turn off this protective measure put in place back in the 3.4 era.
We hope modern kernels are robust enough; if not, NEWS contains
workaround instructions, and we can snap this back on in staprun.
Lukas Berk [Fri, 23 Dec 2016 23:12:01 +0000 (18:12 -0500)]
java: add basic support for java objects
Previously stap was only able to probe primitive java types.
Instead of falling back on string cases if a primitive isn't found
(thus erroring out if the type doesn't match, such as an object), we
now explicitly check for the a string type, and default to converting
the object name to a string if no primitive is found and reporting that.
Originally reported: https://sourceware.org/ml/systemtap/2016-q4/msg00125.html
David Smith [Wed, 21 Dec 2016 16:05:36 +0000 (10:05 -0600)]
Handle python2 class variable references, like '$class->member'.
* tapset-python.cxx (python_var_expanding_visitor::visit_target_symbol):
Rework the python variable handling to support python class variable
references, like '$class->member'.
* tapset/python2.stp (Py2Tuple_Size): New function.
(Py2Class_Lookup): Ditto.
(Py2Class_GetAttr): Ditto.
(Py2Class_Repr): Ditto.
(Py2Instance_GetAttr): Ditto.
(Py2Instance_Repr): Ditto.
(Py2Object_GetItem): Ditto.
(Py2Object_GetAttr): Ditto.
(python2_get_var_obj): Renamed from python2_get_variable(), since it no
longer returns a string representation of a object, but an object
pointer.
* testsuite/systemtap.python/python2.exp: Added class member variable test.
* testsuite/systemtap.python/class.stp: Ditto.
David Smith [Mon, 19 Dec 2016 18:58:00 +0000 (12:58 -0600)]
Handle probes in python class methods.
* python/HelperSDT/__init__.py: Rework breakpoint logic a bit to handle
breakpoints in python classes.
* testsuite/systemtap.python/python2.exp: Add class backtrace test.
* testsuite/systemtap.python/python3.exp: Ditto.
* testsuite/systemtap.python/class.py: New file.
* testsuite/systemtap.python/class.stp: New file.
David Smith [Thu, 15 Dec 2016 19:09:49 +0000 (13:09 -0600)]
Add a small runtime fix to __stp_time_local_update().
* runtime/time.c (__stp_time_local_update): Call
write_seqlock_irqsave()/write_sequnlock_irqrestore() instead of
write_seqlock()/write_sequnlock() to correctly wait while interrupts are
disabled.
Josh Stone [Wed, 14 Dec 2016 19:55:07 +0000 (11:55 -0800)]
Simplify embedded-tags visitors
Both embeddedcode and embedded_expr now have tagged_p members, and a few
of the cases checking for embedded tags can now do so more directly,
rather than trying a deep traversal.
David Smith [Tue, 13 Dec 2016 22:55:36 +0000 (16:55 -0600)]
Fix get_user_pages_remote() autoconf test for 4.8 kernels.
* runtime/linux/autoconf-get_user_pages_remote-flags.c: Since later
autoconf-style test don't get the results of previous tests, remove
STAPCONF_GET_USER_PAGES_REMOTE check.
David Smith [Mon, 12 Dec 2016 19:16:23 +0000 (13:16 -0600)]
Fix python probe script hashing.
* tapset-python.cxx (python_derived_probe_group::enroll): Create/update
the global embedded code that contains the python probe data. This data
appears in pass 2 output and so contributes to the script hash.
Cody Santing [Fri, 9 Dec 2016 21:40:10 +0000 (16:40 -0500)]
PR20889: metadatabase.db location
* Makefile.am: Remove DOCDIR. Copy examples to $pkgdatadir rather than $docdir.
* Makefile.in: Updated using autoreconf.
* NEWS: Mention new examples location.
* doc/Makefile.in: Updated using autoreconf.
* doc/beginners/Makefile.in: same
* interactive.cxx: Look for metadatabase.db under PKGDATADIR rather than DOCDIR.
* java/Makefile.in: Updated using autoreconf.
* systemtap.spec: Update location of examples directory.
David Smith [Tue, 6 Dec 2016 16:54:14 +0000 (10:54 -0600)]
Fix a problem with multiple python probes.
* tapset-python.cxx (python_builder::build): Fix problem of more than one
python probe getting a compile error. The base probe was getting
modified more than once.
David Smith [Fri, 2 Dec 2016 16:45:45 +0000 (10:45 -0600)]
Improve python probe listing.
* python/stap-resolve-module-function.py (_find_wildcarded_modules): Just
ignore C builtin modules and C extensions instead of raising errors.
(resolve_patterns): Don't clear the returned list of matches between
wildcards.
Cody Santing [Thu, 1 Dec 2016 19:46:14 +0000 (14:46 -0500)]
socket-events.stp example revamp WIP
* testsuite/systemtap.examples/general/socket-events.stp: Made the output much more compact. Still has a few things needing to be updated, mainly occasional timing inaccuracies.
David Smith [Thu, 1 Dec 2016 18:55:25 +0000 (12:55 -0600)]
Fix listing python probes.
* tapset-python.cxx (python_builder::build): Fix listing python probes by
marking the new location as well formed and using it in all the correct
places.
David Smith [Wed, 30 Nov 2016 16:53:38 +0000 (10:53 -0600)]
Add support for python target variable numeric array indexing.
* tapset-python.cxx (python_var_expanding_visitor::visit_target_symbol):
Add support for numeric array indexing.
* tapset/python2.stp (Py2Dict_GetItem_FromLong): New function.
(Py2Object_Repr): Add variant of Py2Object_Repr() that takes a numeric
index.
(python2_get_variable): Add variant of python2_get_variable() that takes
a numeric index.
* tapset/python3.stp (Py3Dict_GetItem_FromLong): New function.
(Py3Object_Repr): Add variant of Py3Object_Repr() that takes a numeric
index.
(python3_get_variable): Add variant of python3_get_variable() that takes
a numeric index.
David Smith [Tue, 29 Nov 2016 21:29:09 +0000 (15:29 -0600)]
Add support for getting simple python target variables in probes.
* tapset-python.cxx (python_var_expanding_visitor::visit_target_symbol):
Add support for getting simple python target variables.
(python_builder::build): Add small compile fix for RHEL7 and update
comments.
* tapset/python2.stp (python2_get_variable): New function.
* tapset/python3.stp (Py3Dict_GetItem): Add support for python 3
dictionaries.
(Py3Dict_Repr): Ditto.
(Py3Object_Repr): Simplify and add support for dictionaries.
(python3_get_variable): New function.
* session.cxx (systemtap_session::version): Report python2 and python3
probe support.
* tapset/linux/syscalls.stpm (__pointer): On 32-bit platforms, return a
32-bit value.
Martin Cermak [Tue, 29 Nov 2016 13:33:05 +0000 (14:33 +0100)]
Fix PR20820 by making optim_stats.exp smaller.
Avoid "soft lockups" generated by optim_stats.exp by lowering
its cpu load down. Avoid division by zero caused by rounding
errors by numerical scaling. Introduce per-arch tresholds.
Move the testcase description from optim_stats1.stp to
optim_stats.exp.
Martin Cermak [Wed, 23 Nov 2016 18:45:10 +0000 (19:45 +0100)]
Fix the boot time probing feature for RHEL6 and Fedora19+ systems.
Before this fix, the boot time probing feature was proved working
on RHEL7. This patch adds a few little fixes making the feature
work on RHEL6 and Fedora19+ systems. Changes:
configure.ac: Make the dracut bin dir configurable at the configure
time. This is because of Fedora systems that have the dracut binary
located in /usr/bin rather than /sbin which is the typical location
for RHEL. Also mark initscript/99stap/{install,check} as AC config
files.
initscript/99stap/{install,check}.in: These files are bodies of the
install() and check() functions defined in module-setup.sh extracted
into standalone executable shell scripts. The older dracut shipped
with RHEL6 expects these bits right there.
initscript/99stap/start-staprun.sh: Don't create the pid file at this
stage. This script runs very early at the "cmdline" dracut hoo time.
At this time, RHEL7 systems offer writable /run directory, so that
the pid file can be stored there, but RHEL6 systems do not have a
writable place like that. Rhel6 only has /dev writable at that
stage, but misusing /dev for the pid file purposes wouldn't be clean.
So we use a mechanism based on /proc cescribed below instead.
initscript/systemtap.in: Don't move over any pid files from
/var/run/systemtap, because we stopped creating pid files at the
boot time per above. Instead we use stap -E to add a probe
advertising the running service in /proc/systemtap.
systemtap.spec: A few simple changes making the boot time probing
feature work on RHEL6 and Fedora19 systems.
David Smith [Mon, 21 Nov 2016 18:11:10 +0000 (12:11 -0600)]
Added support for python dictionaries to tapset/python2.stp.
* tapset/python2.stp (Py2Dict_GetItem): New function.
(Py2DictEntry_Repr): Ditto.
(Py2Dict_Repr): Ditto.
(Py2Object_Repr): Added dict support and simplified a bit.
Frank Ch. Eigler [Thu, 17 Nov 2016 16:52:33 +0000 (11:52 -0500)]
PR20821/PR14924: further gradual deprecation of .return { $var } use
* tapsets.cxx (var_expanding_visitor::visit_defined_op): Don't crash
if something weird is nested beneath @defined(...) that throws a
semantic error.
(visit_target_symbol_saved_return): Use save_and_restore<> object
instead of manual save/restore steps.
(dwarf_derived_probe ctor): Ditto.
(visit_entry_op): Ditto. Also, detect nestedness in @defined()
to trigger operand expansion attempt. This is tricky.
* main.cxx (passes_0_4): If -vvv, then pretty-print the script even
if erroneous, to see aftereffects of optimization, $var expansion
etc.
* staptree.cxx (vardecl::printsig): Some synthetic globals don't have
unmangled names, so print their normal name as a fallback.
* elaborate.cxx (visit_perf_op): note that @defined(@perf("junk"))
returns the literal 1. (Recently, it merely crashed the translator
in a .return probe.)
* testsuite/buildok/fortyfour.stp: New test case.
Martin Cermak [Fri, 18 Nov 2016 15:12:36 +0000 (16:12 +0100)]
Fix PR14924 by warning when not using @entry($var) in a return probe.
tapsets.cxx: Show warning when a $var gets used in a return probe
instead of @return($var). Allow the @defined(@entry()) construct.
tapset/linux/*: Use @entry() in return probes.
testsuite/systemtap.base/at_var.exp: Use -w to suppress the newly added
warning in tests that basically check that $var == @entry($var) in a
return probe.
Frank Ch. Eigler [Thu, 17 Nov 2016 11:12:59 +0000 (06:12 -0500)]
translator pass-3: close auxiliary_outputs[] after we're done with them
Otherwise we hang onto potentially dozens of unnecessary fds into
pass4. It is messy that a translator_output doesn't necessarily take
an fstream, and can write to a generic ostream, so we can only close
in the former case.
Cody Santing [Wed, 16 Nov 2016 18:40:16 +0000 (13:40 -0500)]
Update sample command to take multiple keywords
* NEWS: Added a small mention of the new sample command.
* interactive.cxx: Sample now accepts FTS full-text index query statements rather than just one keyword.
* testsuite/systemtap.examples/examples-index-gen.pl: Update to now use the new virtual table.
* testsuite/systemtap.examples/metadatabase.db: New virtual table created that uses FTS3.
David Smith [Fri, 11 Nov 2016 19:37:59 +0000 (13:37 -0600)]
Add support for getting python 2 and python 3 backtraces.
* tapset-python.cxx
(python_functioncall_expanding_visitor::visit_functioncall): New function.
(python_builder::build): Use the new
python_functioncall_expanding_visitor class to expand/replace python
backtrace requests in the probe body.
* tapset/python2.stp: New file.
* tapset/python3.stp: New file.
Cody Santing [Fri, 11 Nov 2016 16:47:49 +0000 (11:47 -0500)]
New example script socket-events.stp
* general/socket-events.meta: Associated meta file
* general/socket-events.stp: Tracks life cycle and syscall events of all sockets of a specified PID.
* index.html: Automatically updated using perl script to include information on new example.
* index.txt: Same
* keyword-index.html: Same
* keyword-index.txt: Same
* metadatabase.db: Same
David Smith [Thu, 10 Nov 2016 15:59:16 +0000 (09:59 -0600)]
Fix PR20735 by updating kprobes.c to avoid a soft lockup.
* runtime/linux/kprobes.c: Instead of grabbing the module mutex around
calling kallsyms_on_each_symbol(), just disable/enable preemption
instead to avoid a soft lockup.