Felix Lu [Tue, 9 Aug 2016 19:36:14 +0000 (15:36 -0400)]
filename based auto path tapset support
Sometimes the process name may change with new versions and the
tapset must be modified. Process probe points in the PATH directory
may now have their arguments replaced by the full file path starting
from /PATH.
* parse.cxx: Pass alias_seen flag to parse_probe_points(). Add filepath
argument to process component.
Felix Lu [Mon, 8 Aug 2016 18:48:37 +0000 (14:48 -0400)]
some function overloading optimizations
This removes some unnecessary code in the generated module.
* elaborate.cxx: Store next flag in functiondecl instead of in a set.
* staptree.h: struct functiondecl:: new has_next flag.
* translate.cxx: c_unparser::visit_functioncall - Don't generate
redundant functioncall code for overloaded functions.
David Smith [Thu, 4 Aug 2016 20:39:19 +0000 (15:39 -0500)]
Fix a couple of conversions.stp tapset problems and improve its testing.
* tapset/linux/conversions.stp (kernel_buffer_quoted): Properly print the
address as an unsigned long (instead of an unsigned int).
(kernel_buffer_quoted_error): Add the faulting address in the error
message.
* testsuite/buildok/conversions-embedded.stp: Add compile tests for
several missing functions.
Felix Lu [Thu, 28 Jul 2016 14:39:58 +0000 (10:39 -0400)]
PR10485: auto-path tapset support
This prefixes the process probes placed in the tapset/PATH/ directory
with the location of the tapset.
* main.cxx: New collect_{stp,stpm} functions for passing to nftw(). Parse all
.stp files placed in include_path[0]/PATH/. nftw() is now used instead of
globbing for tapset searching to allow recursive traversal.
* parse.h: New pf_auto_path flag.
* parse.cxx: Prefix process argument with location of tapset.
PR20423: improve 'struct {...} being accessed instead of member' message
Esp. for anonymous structures, this message was not very helpful.
The new message is much more complete:
semantic error: 'struct {...}' (include/linux/types.h:175) is being accessed instead of a member such as '->counter': operator '->' at <input>:1:81
source: probe kernel.function("dput") { println( @cast($dentry->d_inode, "struct inode")->i_count) }
^
Igor Zhbanov [Fri, 29 Jul 2016 17:51:21 +0000 (13:51 -0400)]
systemtap.spec: add systemd as BuildRequires & Requires if %{with_systemd}
It should have both "Requires: systemd" and "BuildRequires: systemd".
BuildRequires because it uses %{_unitdir} and %{_tmpfilesdir}, and
Requires because of systemctl in %post.
The dwarf_var_expanding_visitor::visit_entry_op function inherited
some old logic from expanding $var's in .function().return probes.
This logic is not approproate for the more formal way by which current
code synthesizes a .call probe to collect @entry(EXPR) values. Now we
pass the EXPR tree to the .call probe unmodified, for resolution
there, rather than in a fake non-.return context here. This makes
@entry(@perf("...")) expressions work.
David Smith [Tue, 26 Jul 2016 17:57:59 +0000 (12:57 -0500)]
Fix a locking problem in the task_finder.
* runtime/linux/task_finder.c (stap_start_task_finder): Be sure to ulock
the task if we exit early.
* runtime/linux/task_finder2.c (stap_start_task_finder): Ditto.
David Smith [Mon, 25 Jul 2016 15:24:27 +0000 (10:24 -0500)]
Improve session state and transport state handling.
* translate.cxx (emit_module_init): At the start, only set the session
state to STAP_SESSION_STARTING if we were in the
STAP_SESSION_UNINITIALIZED state. Once the session is initialized, set
the state to STAP_SESSION_RUNNING (if we're still in the
STAP_SESSION_STARTING state).
* runtime/common_session_state.h: Add new 'STAP_SESSION_UNITIALIZED' state.
* runtime/dyninst/common_session_state.h (stp_session_init): Set initial
session state to STAP_SESSION_UNITIALIZED.
* runtime/linux/common_session_state.h: Ditto.
* runtime/dyninst/linux_defs.h (atomic_cmpxchg): New function.
* runtime/transport/transport.c: Merge '_stp_start_called' and
'_stp_exit_called' flags into a single flag called
'_stp_transport_state'.
Martin Cermak [Thu, 21 Jul 2016 16:17:16 +0000 (18:17 +0200)]
Don't use as_root() to run stap within tapset_functions.exp
Most of the other as_root() in the testsuite is for stuff that
genuinely has to be root, like loading some test kernel module
that we'll probe. For running the testsuite under a regular
user, stap{user,sys,dev} group membership mechanism should be
used instead.
David Smith [Fri, 15 Jul 2016 16:14:21 +0000 (11:14 -0500)]
Fix handling of internal transport flags.
* runtime/transport/transport.c: Remove unused '_stp_transport_mutex'
mutex. Commit 440c8ee removed the mutex protection of the
'_stp_start_called' and '_stp_exit_called' flags, so those flags have
been converted to be atomics so that concurrent use of those flags see
the correct values.
David Smith [Fri, 15 Jul 2016 16:00:33 +0000 (11:00 -0500)]
Make sure the unprivileged_myproc.exp testcase kills its processes.
* testsuite/systemtap.unprivileged/unprivileged_myproc.exp: Use library
'kill' proc to make sure the test executables die.
* testsuite/lib/systemtap.exp (kill): Add 'AS_ROOT' argument, to kill the
pid as root.
Josh Stone [Sat, 9 Jul 2016 01:21:49 +0000 (18:21 -0700)]
Tweak autocast-defined interactions further
- collapse basic @defined($foo) right away.
- last-ditch collapse other @defined(expr) to 1 or 0 depending on pe_unknown.
- run that last-ditch effort *before* turning on assert_resolvability.
- only run extra dead_control_remover for optimized runs
- in var_expanding_visitor, pass *any* unchanged expr through, so they
may be decided later. (e.g. for @choose_defined ternaries)
Frank Ch. Eigler [Wed, 22 Jun 2016 15:43:33 +0000 (11:43 -0400)]
PR18079: support nested autocast / @defined
We now perform const-folding & dead-code-elision during the type
resolution loop, whenever an autocast expression gets evaluated. This
way, @defined(foo()->mm) type expressions can work as nature intended.
This requires @defined() not to be short-circuit evaluated to 0 during
a random const_folding process, so a flag is introduced to control its
preservation or collapsing. For the last (assert_resolvability) pass
in the type resolution loop, this flag is set to true, so that
genuinely unresolvable @defined($expressions) do get mapped to 0 in
time for a last elision.
Martin Cermak [Thu, 21 Jul 2016 16:17:16 +0000 (18:17 +0200)]
Don't use as_root() to run stap within tapset_functions.exp
Most of the other as_root() in the testsuite is for stuff that
genuinely has to be root, like loading some test kernel module
that we'll probe. For running the testsuite under a regular
user, stap{user,sys,dev} group membership mechanism should be
used instead.
David Smith [Fri, 15 Jul 2016 16:14:21 +0000 (11:14 -0500)]
Fix handling of internal transport flags.
* runtime/transport/transport.c: Remove unused '_stp_transport_mutex'
mutex. Commit 440c8ee removed the mutex protection of the
'_stp_start_called' and '_stp_exit_called' flags, so those flags have
been converted to be atomics so that concurrent use of those flags see
the correct values.
David Smith [Fri, 15 Jul 2016 16:00:33 +0000 (11:00 -0500)]
Make sure the unprivileged_myproc.exp testcase kills its processes.
* testsuite/systemtap.unprivileged/unprivileged_myproc.exp: Use library
'kill' proc to make sure the test executables die.
* testsuite/lib/systemtap.exp (kill): Add 'AS_ROOT' argument, to kill the
pid as root.
PR19489: Update send syscalls to use buffer_quoted
tapset/linux/nd_syscalls2.stp: send and sendto use new user_buffer_quoted function
tapset/linnux/syscalls2.stp: send and sendto use new user_buffer_quoted function
testsuite/systemtap.syscall/send.c: Update expected output of testcases
testsuite/systemtap.syscall/sendto.c:Update expected output of test cases
Josh Stone [Sat, 9 Jul 2016 01:21:49 +0000 (18:21 -0700)]
Tweak autocast-defined interactions further
- collapse basic @defined($foo) right away.
- last-ditch collapse other @defined(expr) to 1 or 0 depending on pe_unknown.
- run that last-ditch effort *before* turning on assert_resolvability.
- only run extra dead_control_remover for optimized runs
- in var_expanding_visitor, pass *any* unchanged expr through, so they
may be decided later. (e.g. for @choose_defined ternaries)
PR19489: Update probe aliases to use new *buffer_quoted functions
tapset/linux/nd_syscalls.stp: add_key uses new function
tapset/linux/nd_syscalls2.stp: pwrite, pwrite32, and write use new function
tapset/linux/syscalls.stp: add_key uses new function
tapset/linux/syscalls2.stp: pwrite and pwrite32 use new function
tapset/linux/tty.stp: write and read use new function
tapset/uconversions.stp: Update typo in documentation comments
Fix cputime_to_msecs() macro for broken android 3.0 kernels.
* tapset/linux/task_time.stp (cputime_to_msecs): Fix macro definition for
certain android kernels that have an extra semicolon on the
cputime_to_usecs() macro.
Felix Lu [Thu, 30 Jun 2016 18:54:18 +0000 (14:54 -0400)]
PR19953: netfilter packet content dumping
* tapset/linux/conversions.stp: New overloaded instance of
kernel_buffer_quoted using MAXSTRINGLEN by default.
Mark kernel_buffer* with /* pure */.
* tapset/linux/netfilter.stp: New data_str and data_hex variables
containing buffer contents.
* translate.cxx: Increase memory dump limit to PAGE_SIZE for memory
dumping sk_buff page fragments in c_unparser::visit_print.
* systemtap.examples/network/packet_contents.stp: Example script.
* systemtap.examples/network/packet_contents.txt: Example demo.
PR19489: Update user/kernel_buffer_quoted_warn tapset functions to _error
/NEWS: Include information on new functions
/tapset/linux/aux_syscalls.stp: Uses new buffer mode of _stp_text_str
/tapset/linux/conversions.stp: Update kernel_buffer_quoted_warn to _error
/tapset/uconversions.stp: Updated user_buffer_quoted_warn to _error
runtime/stp_string.c: Update _stp_text_str to handle '\0' and take a buffer flag
runtime/stp_string.h: Update _stp_text_str prototype
tapset/linux/aux_syscalls.stp: Update calls to _stp_text_str
tapset/linux/conversions.stp: Add user_buffer_quoted and _warn tapset functions
tapset/linux/syscalls2.stp: Update syscall.write to use user_buffer_quoted
tapset/string.stp: Update calls to _stp_text_str
tapset/uconversions.stp: Add kernel_buffer_quoted and _warn tapset functions
Frank Ch. Eigler [Thu, 30 Jun 2016 16:08:07 +0000 (12:08 -0400)]
PR20307: fix references to private global in tapset
Previous code could not resolve a reference to a private global
(defined in a tapset) within a probe alias (from that same tapset).
We now try matching up the mangled name in addition to the plain one
in symresolution_info::find_var() when searching the
session.library_files[].globals.
David Smith [Tue, 28 Jun 2016 15:12:25 +0000 (10:12 -0500)]
Partial 20236 fix: Add user_TYPE_error functions that throw errors.
* tapset/uconversions.stp: Add new functions 'user_TYPE_error()' (where
TYPE is 'char', 'short', 'ushort', 'int', 'long', 'ulong', 'int8',
'uint8', 'int16', 'uint16', 'int32', 'uint32', or 'int64'). These
funtions thow an error if the value to be read isn't
accessible. Reimplement 'user_TYPE()' and 'user_TYPE_warn()' functions
in terms of user_TYPE_error() functions.
* tapset/linux/aux_syscalls.stp: Replace '__user_pointer' with
'user_long_error'.
* testsuite/buildok/conversions-embedded.stp: Add tests for new functions.
Martin Cermak [Tue, 28 Jun 2016 14:25:33 +0000 (16:25 +0200)]
Don't error on missing systemtap.sum
Don't report error if systemtap.sum isn't available to tail.
This was producing optically disturbing error message e.g. in case
the testsuite got run without dejagnu installed.
Martin Cermak [Tue, 28 Jun 2016 12:21:04 +0000 (14:21 +0200)]
Implement PR12748 by providing the syscall_table tapset files.
This patch provides architecture specific syscall tables, tapset
script functions syscall_num() and syscall_name() for acessing
them, test coverage, and a script for re-generating the syscall
tables based on strace source code.
* scripts/dump-syscalls.sh: Helper script for re-generating the
syscall tables based on strace source code.
* tapset/linux/syscall_table.stp: Tapset script providing
functions syscall_num() and syscall_name().
* tapset/linux/arm/syscall_num.stp: Syscall table.
* tapset/linux/arm64/syscall_num.stp: Ditto.
* tapset/linux/i386/syscall_num.stp: Ditto.
* tapset/linux/powerpc/syscall_num.stp: Ditto.
* tapset/linux/s390/syscall_num.stp: Ditto.
* tapset/linux/x86_64/syscall_num.stp: Ditto.
* testsuite/systemtap.syscall/sys.stp: A testsuite bit.
* testsuite/systemtap.syscall/syscall.exp: Ditto.
* testsuite/systemtap.syscall/tapset/syscall.stp: Ditto.
David Smith [Mon, 27 Jun 2016 17:57:05 +0000 (12:57 -0500)]
Fix PR20298 by updating the unprivileged_embedded_C.exp test case.
* testsuite/systemtap.unprivileged/unprivileged_embedded_C.exp: Several
changes. Now only tests embedded C functions (since we can't really know
if pure script functions are privileged or not). Look for tapset files
in tapset/linux and in tapset/linux/${ARCH}. Now test all the embedded C
functions intead of a subset. Add a "transitive" test, to make sure
privileged embedded C functions can't be called by script functions.
Nikolay Borisov [Mon, 27 Jun 2016 16:46:35 +0000 (19:46 +0300)]
Add the '-p4' options when exemplifying the module compilation
Currently the example command which supposedly should compile the
instrumentation module is missing the '-p4' option, meaning that
upon running it will compile and run the module. In order to make
the command more in sync with what this particular chapter is about,
add the -p4 options so that after running the command the user gets
to copy the resulting module.
Signed-off-by: Nikolay Borisov <n.borisov.lkml@gmail.com>
Frank Ch. Eigler [Wed, 22 Jun 2016 15:43:33 +0000 (11:43 -0400)]
PR18079: support nested autocast / @defined
We now perform const-folding & dead-code-elision during the type
resolution loop, whenever an autocast expression gets evaluated. This
way, @defined(foo()->mm) type expressions can work as nature intended.
This requires @defined() not to be short-circuit evaluated to 0 during
a random const_folding process, so a flag is introduced to control its
preservation or collapsing. For the last (assert_resolvability) pass
in the type resolution loop, this flag is set to true, so that
genuinely unresolvable @defined($expressions) do get mapped to 0 in
time for a last elision.
Felix Lu [Tue, 21 Jun 2016 20:47:24 +0000 (16:47 -0400)]
Improve error messages
List the probe builders tried during resolution.
* dwflpp.cxx: Display error message when using wildcard for linenumber
mode if no matches.
* elaborate.cxx:
- match_node::find_and_build: Accumulate builders tried
when no probes were resolved.
- derive_probes: Display builders in error chain.
* elaborate.h: New name() for derived_probe_builder.
Martin Cermak [Tue, 21 Jun 2016 14:07:02 +0000 (16:07 +0200)]
Testsuite resume mode only works with its own files.
This update makes the testsuite resume mode only work with its own
files, not touching logs that the user might have set aside for
his/her own reference in the testsuite directory. After this
update, the log names "reserved" for the testsuite resume mode
functionality are systemtap.temp{1,2}.{log,sum}.
David Smith [Fri, 17 Jun 2016 14:16:15 +0000 (09:16 -0500)]
Partial 20236 fix: Cleanup memory accesses and always call lookup_bad_addr().
* runtime/linux/loc2c-runtime.h: Fairly serious rewrite of the
deref/store_deref macros. Instead of custom assembly language macros for
each arch, now we just use the kernel's routines.
(__stp_deref_nocheck): New macro.
(_stp_deref_nofault): New macro.
(__stp_store_deref_nocheck): New macro.
(_stp_store_deref_nofault): New macro.
(kderef_buffer): Rewrote using lookup_bad_addr() and
__stp_deref_nocheck(), allowing us to do 1 address check at the start of
the function instead of checking byte by byte.
(kderef_string): Ditto.
(_stp_store_deref_string): New macro.
(store_kderef_string): Rewrite in terms of _stp_store_deref_string().
(store_uderef_string): Ditto.
* runtime/linux/copy.c (_stp_strncpy_from_user): Always use the kernel's
strncpy_from_user(). Call lookup_bad_addr() on target address.
(_stp_copy_from_user): Call lookup_bad_addr() on target address.
* runtime/stp_string.h (_stp_read_address): Delete
macro. _stp_deref_nofault() can be used instead.
* runtime/stp_string.c (_stp_decode_utf8): Use new macro
_stp_deref_nofault() instead of _stp_read_address().
* runtime/unwind.c: Ditto.
* runtime/unwind/unwind.h (read_ptr_sect): Ditto.
* runtime/addr-map.h: New file.
* runtime/addr-map.c: Include addr-map.h.
* runtime/runtime.h: Include addr-map.h so that early code can use
lookup_bad_addr().
* runtime/linux/addr-map.c (lookup_bad_addr): Updated prototype.
* runtime/dyninst/addr-map.c (lookup_bad_addr): Update function
prototype. Add copyright header.
* runtime/dyninst/loc2c-runtime.h (_stp_deref_nofault): New macro to match
linux runtime changes.
Josh Stone [Thu, 16 Jun 2016 23:32:50 +0000 (16:32 -0700)]
testsuite: refactor pthreadprobes.exp
- move the main script into its own pthreadprobes.stp
- reduce repetition with macros, aliases, and an array of counters
- suppress warnings from stapdyn so individual failures can be checked
David Smith [Thu, 16 Jun 2016 17:26:06 +0000 (12:26 -0500)]
Make some small improvements to [nd_]syscall.fanotify_mark for RHEL7.
* tapset/linux/syscalls.stp (syscall.fanotify_mark): Call
__syscall_gate_compat_simple to avoid compat calls in the real syscall.
* tapset/linux/nd_syscalls.stp (nd_syscall.fanotify_mark): Ditto.
* testsuite/systemtap.syscall/fanotify.c: Make the fanotify file
descriptor non-blocking, to avoid the testcase hanging on read.
Cody Santing [Wed, 15 Jun 2016 13:47:50 +0000 (09:47 -0400)]
PR11637: Completed test cases, updated documentation
/NEWS: Updated NEWS to include information about set_user_*() functions
/doc/SystemTap_Tapset_Referemce/tapsets.tmpl: Include new functions in tapsets documentation
/testsuite/systemtap.base/set_user.c: Generates user space address for use in test case
/testsuite/systemtap.base/set_user.stp: Updated test case to test all functions
/testsuite/systemtap.base/set_user_fail.stp: Removed test case
David Smith [Tue, 14 Jun 2016 13:34:04 +0000 (08:34 -0500)]
Partial 20236 fix: Add access_ok() to lookup_bad_addr().
* runtime/linux/addr-map.c (lookup_bad_addr): Now calls access_ok(), which
required the addition of a 'type' argument. In addition, there are now 2
versions of lookup_bad_addr(), the new one doesn't bother searching the
bad address database, since we don't populate that yet.
* runtime/dyninst/addr-map.c (lookup_bad_addr): Added dummy 'type'
argument.
* runtime/linux/loc2c-runtime.h: Add either VERIFY_READ or VERIFY_WRITE to
every lookup_bad_addr() call.
David Smith [Fri, 10 Jun 2016 19:52:57 +0000 (14:52 -0500)]
Update 4 syscall tests for rawhide.
* testsuite/systemtap.syscall/recvmmsg.c: Skip a test that causes a
SEGFAULT on rawhide's glibc.
* testsuite/systemtap.syscall/recvmsg.c: Ditto.
* testsuite/systemtap.syscall/sendmsg.c: Ditto.
* testsuite/systemtap.syscall/sendmmsg.c: Create our own version of
sendmmsg(), since rawhide's glibc likes to substitute sendmsg() for
sendmmsg().
Martin Cermak [Wed, 8 Jun 2016 10:47:00 +0000 (12:47 +0200)]
Fix PR20211 by aimplementing the "resume mode" of the testsuite.
Once the SYSTEMTAP_TESTSUITE_RESUME environmental variable is set
to some non-zero value, the testsuite can run in "resume mode".
In this mode, tests, that have previously been run, are not being
re-executed, and their existing test results are being reused and
merged instead.
This can be useful e.g. when some of the tests causes kernel panic
or stall. Using the "resume mode", it is possible to reboot the
testing box, and re-run the testsuite skipping already executed
tests (incl. the one that caused the panic or stall - which will
stay "unfinished").
The "make list-unfinished" lists all such unfinished testcases.
Those need additional investigation.
The "resume mode" can be used with "make installcheck", or
"make installcheck-parallel".
Cody Santing [Tue, 7 Jun 2016 15:29:12 +0000 (11:29 -0400)]
11637: Added test cases, added to set_user_* functions
tapset/uconversion-guru.stp : Added deref_fault handlers, /* guru */ tag as well as comments.
testsuite/systemtap.base/set_user.c: Used to generate user space address
testsuite/systemtap.base/set_user.exp: Added to incorporate into testsuite
testsuite/systemtap.base/set_user.stp: Calls set_user_* functions and confirms values written to correct address
testsuite/systemtap.base/set_user_fail.stp: Attempt to write to kernel space address
Martin Cermak [Fri, 27 May 2016 19:41:49 +0000 (21:41 +0200)]
Re-fix PR19926 by simplifying previous implementation.
This reimplementation of the @const() operator (0a7eb12d191)
is based on PR20136, comment 3. It moves all the logic to the
parser. No elaboration involved.