PR14245 created a new relay_basedir_fd variable, for use when
staprun/stapio passes staprun a file descriptor for the base directory
of the sysfs systemtap-module directory. (This is necessary in order
to get around 0700 mount-dir permissions.) This broke staprun -d '*'
operation, since that variable got set (badly) once within the readdir
loop, and thence made actual removes inoperative. Now we clear out
that variable so each actual module subdirectory will get a good
cleaning.
David Smith [Mon, 25 Apr 2016 15:02:36 +0000 (10:02 -0500)]
Fix PR19990 by updating runtime/linux/access_process_vm.h.
* runtime/linux/access_process_vm.h (__access_process_vm_): Use
get_user_pages_remote() when available.
* buildrun.cxx (compile_pass): Added export test for
'get_user_pages_remote()'.
William Cohen [Fri, 15 Apr 2016 19:15:22 +0000 (15:15 -0400)]
Avoid using tcl exec command "-ignorestderr" option
In older versions of tcl the exec command does not support the
"-ignorestderr" option. Changed the code to redirected the stderr
output to /dev/null which works on both older and newer versions of
tcl.
David Smith [Wed, 13 Apr 2016 15:11:17 +0000 (10:11 -0500)]
Remove a blacklist regular expression that wasn't needed.
* dwflpp.cxx (build_kernel_blacklist): Removed a blacklist regexp,
"|raw_.*", since it didn't seem to match any locking functions as it was
supposed to do.
Martin Cermak [Wed, 13 Apr 2016 13:31:33 +0000 (15:31 +0200)]
Move the testsuite's artifacts directory to the builddir.
This update is a response to PR10791#c15. It moves the artifacts
directory, introduced in commit 8093ef39b, to the build directory
avoiding writes to the source directory.
Martin Cermak [Wed, 13 Apr 2016 13:22:10 +0000 (15:22 +0200)]
Modify the dry run mode for the network/socktop example.
We want to avoid testsuite writing into the source directory.
Before this update, when socktop has been run by check.exp as
a part of the testsuite run, it was copying the module to the
working directory. We want to avoid this.
Martin Cermak [Mon, 11 Apr 2016 07:55:20 +0000 (09:55 +0200)]
Terminate testsuite run if compiler sanity test fails.
Ensure that the testsuite run terminates right at the beginning
if compiler sanity test fails. This notably ensures that required
headers, object files and libraries are in place for the main arch,
as well as for the compat arch in the multilib environment.
David Smith [Fri, 8 Apr 2016 18:40:04 +0000 (13:40 -0500)]
Improve 64-bit file offset handling in syscall.preadv and syscall.pwritev.
* tapset/linux/syscalls2.stp (syscall.preadv): Improve handling of 64-bit
offsets.
(syscall.pwritev): Ditto.
* testsuite/systemtap.syscall/preadv.c: Turn on 64-bit file offsets and
add tests to make sure we're decoding them properly.
* testsuite/systemtap.syscall/pwritev.c: Ditto.
Vincent Bernat reported problems building with --enable-htmldocs due
to the newly overloaded ctime() function docs. Tweak the /** */
docs for the second overload, so as to suffer errors in neither
html nor pdf build phases.
Abegail Jakop [Tue, 29 Mar 2016 23:03:32 +0000 (19:03 -0400)]
PR19882: add copy_file_range syscall supportt
tapset/linux/nd_syscalls.stp: kprobe based syscall probe for
copy_file_range syscall
tapset/linux/syscalls.stp: kernel function based syscall probe for
copy_file_range syscall
buildok/[nd_]syscalls-detailed.stp: basic buildok tests for the new
probe points
copy_file_range.c: test file to check new tapset functionality
David Smith [Wed, 30 Mar 2016 14:49:30 +0000 (09:49 -0500)]
Cleaned up 'pure' and 'unprivileged' annotations in aux_syscalls.stp.
* tapset/linux/aux_syscalls.stp: Cleaned up. Marked _membarrier_cmd_str()
as a string function. Audited all functions and constants to make sure
they are marked as 'pure' and 'unprivileged' when needed.
David Smith [Tue, 29 Mar 2016 19:53:20 +0000 (14:53 -0500)]
Fix PR19876 by adding tapset support for the 'userfaultfd' system call.
* tapset/linux/syscalls2.stp: Add support for the 'userfaultfd' system
call.
* tapset/linux/nd_syscalls.stp: Ditto.
* tapset/linux/aux_syscalls.stp (_userfaultfd_flags_str): New function.
(_sys_pipe2_flag_str): Added O_DIRECT to the list of supported flags.
* testsuite/buildok/syscalls2-detailed.stp: Add userfaultfd test.
* testsuite/buildok/nd_syscalls2-detailed.stp: Ditto.
* testsuite/systemtap.syscall/userfaultfd.c: Add new
syscall.exp/nd_syscall.exp test case.
David Smith [Tue, 29 Mar 2016 16:29:40 +0000 (11:29 -0500)]
Fix PR19875 by adding tapset support for the 'membarrier' system call.
* tapset/linux/syscalls.stp: Add support for the 'membarrier' system
call.
* tapset/linux/nd_syscalls.stp: Ditto.
* tapset/linux/aux_syscalls.stp (_membarrier_cmd_str): New function.
* testsuite/buildok/syscalls-detailed.stp: Add membarrier test.
* testsuite/buildok/nd_syscalls-detailed.stp: Ditto.
* testsuite/systemtap.syscall/membarrier.c: Add new
syscall.exp/nd_syscall.exp test case.
David Smith [Tue, 29 Mar 2016 16:25:01 +0000 (11:25 -0500)]
Fix the interactive stap's buildok-interactive.exp test case.
* testsuite/lib/stapi.exp (stapi_start): Set the interactive stap's
termibnal to 4096 columns, to avoid problems with long pathnames needing
to scroll.
* testsuite/systemtap.pass1-4/buildok-interactive.exp: Tweak switch
statements so that tcl will know when the arguments stop. This was
bothering RHEL5's ancient tcl (8.4.3).
Frank Ch. Eigler [Mon, 28 Mar 2016 00:33:04 +0000 (20:33 -0400)]
pre-release docs rebuild
Updated the tapset-doc-filter overload.py to more accurately
process overloaded entities. One should count by 'programlisting'
nodes, not by 'para', which can repeat within any single overload.
Frank Ch. Eigler [Sun, 27 Mar 2016 18:45:01 +0000 (14:45 -0400)]
PR19873: staprun -o BADFILE -c CMD cleanup
* staprun/mainloop.c (start_cmd): Close fd >= 3 to ensure no dangling
reference to the stap module /.cmd file. Set an alarm timeout, in
case the parent process goes away and forgets to send us our SIGUSR
to exec the CMD.
Frank Ch. Eigler [Sun, 27 Mar 2016 18:42:00 +0000 (14:42 -0400)]
testsuite examples check.exp: Don't try to clean up output files.
Scraping file name patterns out of a test_installcheck command line
is fragile, undocumented, and unnecessary. Plus if the code failed,
it could error out of the whole .exp file. Stop trying to clean
this up.
Frank Ch. Eigler [Sat, 26 Mar 2016 00:44:43 +0000 (20:44 -0400)]
PR19833: turn cleanup_module / init_module inside out
Sort of ... at least, make the synthetic systemtap_kernel_module_{init,exit}
unctions the outermost rather than innermost wrapping layer. This is because
in the procfs case, the synthetic _exit function can run first (if the
module's being rmmod'd without an STP_EXIT message), and the subsequent
transport / belated-stap-probe-cleanup code can run second. That's bad
if the former tries to nuke /proc/systemtap/$DIR while the latter still
has subdirectories/files it wants to clean up (such as for procfs.read
objects).
So now we make the /proc/systemtap/$DIR [de]construction be fairly far out.
Frank Ch. Eigler [Fri, 25 Mar 2016 00:20:20 +0000 (20:20 -0400)]
translate.cxx monitor_status probe: add try/catch
Tolerate overflow of the procfs output buffer used for collecting the
monitor_status JSON content. Also, come up with a reasonable WAG for
how much data we might write. It could be far more than 8K if we're
talking about scripts with hundreds of probes.
Frank Ch. Eigler [Fri, 25 Mar 2016 00:19:27 +0000 (20:19 -0400)]
monitor mode: show some status even for exited script
Retain the parsed json object from refresh to refresh, so in case
the module exits, we still can draw a curses window from the last
available content. (Print a reasonable error message otherwise.)
Frank Ch. Eigler [Wed, 23 Mar 2016 22:02:15 +0000 (18:02 -0400)]
stapio main loop: always time out on pselect()
The main thread's pselect() call must now get a timeout setting too,
because it is responsible for orderly shutdown of the stap module in
case of interrupts. Without a polling timeout, the pselect() can just
sit there, with other reader threads happily continuing their work,
as though the ^C never took effect.
David Smith [Wed, 23 Mar 2016 20:44:10 +0000 (15:44 -0500)]
Fix staprun compile error when HAVE_MONITOR_LIBS isn't defined.
* staprun/common.c (monitor_winch): Add empty function when
HAVE_MONITOR_LIBS isn't defined, to make the compiler happy. This is
necessary since the function's address is used in setup_main_signals().
* staprun/staprun.h: When HAVE_MONITOR_LIBS isn't defined, change
monitor_winch() from an inline function to a declaration.
William Cohen [Wed, 23 Mar 2016 20:10:17 +0000 (16:10 -0400)]
Avoid "make distclean" removing removing various man pages
The man pages are not generated by the regular "make". They are now
generated using a special step and are now part of the sources. The
"make distclean" should not remove these prebuilt pages.
Frank Ch. Eigler [Wed, 23 Mar 2016 14:48:28 +0000 (10:48 -0400)]
monitor mode: staprun revamp
A group of changes for usability & stability of the staprun side of
--monitor mode. The gist of it is to let staprun persist awhile after
a script exits (whether interactively commanded with 'q', or with the
script exiting due to an exit/error), so that its outputs can be seen.
* monitor.c (monitor_state): Add new "exited" and "exited_help" states.
(monitor_render): Remove code that reads input from module.
Support exited_help subset text.
(monitor_remember_output_line): New utility function to strndup
module output lines into our circular buffer.
(monitor_input): Revamp logic to use read(2) syscalls instead of
temporary fdopen() FILEs to read from module-output pipe, for
several reasons.
(monitor_exited): New function to enter exited state.
* relay.c (reader_thread): Use write(2) loop when feeding output
pipe/file, in case it gets full.
(init_relayfs): Drop F_SETPIPE_SZ; it is not needed with write loop
here and proper nonblocking read on other side.
* mainloop.c (signal_thread): Keep it running after each signal, just
in case a ^C is repeated. Encode SIGQUIT with a set to 'load_only'
instead of pending_interrupts count.
(stp_main_loop): Treat several startup-time errors as worthy of
stap module removal/cleanup rather than detaching. Within main
loop proper, call cleanup_and_exit upon pending_interrupts, as
the single standard spot where a stap module shutdown from any
cause should come. Redirect warning/error messages to monitor
output buffer. Notify monitor code of received STP_EXIT message.
* staprun.h: Provide inline-void stub functions for the !MONITOR_LIBS
case.
David Smith [Mon, 21 Mar 2016 22:08:59 +0000 (17:08 -0500)]
Fix PR17393 by improving the way we execute "system" commands.
* staprun/common.c (open_cloexec): New function.
(openat_cloexec): Ditto.
(closefrom): ditto.
* staprun/staprun.h: Add prototypes for new functions.
* staprun/ctl.c (init_ctl_channel): Before trying to open the control
channel, make sure it isn't open already. Use openat_cloexec() and
open_cloexec() when opening the control channel. Remove unneeded call to
set_clexec().
* staprun/relay.c (open_outfile): Use open_cloexec(). Remove unneeded call
to set_clexec().
(init_relayfs): Ditto.
* staprun/relay_old.c (open_relayfs_files): Ditto.
(init_oldrelayfs): Ditto.
* staprun/mainloop.c (start_cmd): Add comment explaining the closefrom()
isn't needed here.
(system_cmd): Wait until the child has closed everything (by calling
closefrom) before returning.
* staprun/staprun.c (term_signal_handler): Remove unneeded call to
close_ctl_channel(), since remove_module() now calls it.
Felix Lu [Mon, 21 Mar 2016 20:31:07 +0000 (16:31 -0400)]
Monitor mode: clamp probe list scrolling
Previously, the probe list could be scrolled beyond the number of probes
available and display an empty list. This also made it harder to scroll
up. Also, the number of script probes is now extracted using the json list.
Felix Lu [Mon, 21 Mar 2016 14:13:27 +0000 (10:13 -0400)]
Remove monitor mode global mangling
Commit 87103d2b32ea873721 does not mangle variable names
during the search so the synthetic monitor_enabled variables could
not be found. Those synthetic variables are now not mangled to
allow them to be found.
Frank Ch. Eigler [Sun, 20 Mar 2016 23:18:49 +0000 (19:18 -0400)]
monitor mode: add 'q' keystroke to quit
It'll be nice to have a sequence other than ^C to cause an exit, so
that e.g. we can review output / diagnostics after the script quit.
Switching to 'h' to flip between help screen and normal mode, to avoid
accidental 'q'.
Frank Ch. Eigler [Sun, 20 Mar 2016 22:22:01 +0000 (18:22 -0400)]
monitor mode: deconflict with private global mangling
The synthesized probe body for monitor mode control needs to refer to
globals that may be declared private inside a tapset, in order to
clear them. It needs to pierce the mangling-based name-hiding, so
that when the synthetic fragment is symbol-resolved, it will find the
mangled private globals to manipulate. This is implemented by an
optional mode-flag in the symresolution_info class to suppress
global-mangling.
Frank Ch. Eigler [Sun, 20 Mar 2016 22:20:49 +0000 (18:20 -0400)]
translator: PR13283 redux, optimize string comparison again
With the function-overloading machinery now assuring that function
calls translate to full temporaries to hold return values, we no
longer need to make yet another copy just to evaluate string
comparisons.
Felix Lu [Fri, 18 Mar 2016 18:11:05 +0000 (14:11 -0400)]
Add some monitor mode improvements.
Added an option to pause and resume the script while running to
allow the user to review the output. Previously, a synthetic probe
was being displayed in the probe list, this is now removed.
* elaborate.cxx:
- monitor_mode_write: skip generating probe point
condition for synthetic probes. Add command to resume
and pause all probes.
- semantic_pass: call monitor_mode_{read, write} in init
* staprun/monitor.c: Bind keys for pause and resume.
Frank Ch. Eigler [Fri, 18 Mar 2016 01:24:50 +0000 (21:24 -0400)]
PR19802: hashbench.stp
Rework into the beginning of a hash benchmark suite.
The first test is for plotting 1/2/3-d hash value distributions,
with a bunch of modifiable parameters (use stap "-G param=value").
Frank Ch. Eigler [Thu, 17 Mar 2016 20:19:21 +0000 (16:19 -0400)]
stap dyninst: allow -B option to pass CFLAGS
Formerly, -B (session.kbuildflags) was only used during runtime=kernel
builds. Now it can be used to pass CFLAGS to the compiler run that
builds the generated dyninst-bound stap_FOO.so shared library.
Frank Ch. Eigler [Thu, 17 Mar 2016 20:04:51 +0000 (16:04 -0400)]
dyninst runtime: make ilog2 evaluable even with -O0
While debugging a generated dyninst module, it became an obstacle that
the __builtin_constant_p-guarded ilog2() macro expression failed down
to a ____ilog2_NaN dummy function. With -O0, that function is at least
referenced, thus blocking linkage. We ditch the constant-p checking,
and provide an abort()ing fallback.
Frank Ch. Eigler [Thu, 17 Mar 2016 19:25:19 +0000 (15:25 -0400)]
PR19802: inline murmurhash3 into map-gen.h
Instead of calling into the distributed murmurhash3 function, break it
up so that the map-gen.h hash function inlines it. Each index
(integer or string) is mixed into the hash naturally as though it were
copied into one large contiguous block.
The results show very good dispersion when compared with the kernel
hash_64() function. Evaluating the hash function is somewhat slower
(30%-ish). (Dumbing it down a little bit was attempted a couple of
ways; none were superior enough in dispersion/performance.)
Frank Ch. Eigler [Mon, 14 Mar 2016 01:33:51 +0000 (21:33 -0400)]
PR19802 prototype: switch to murmurhash3
Instead of the kernel's hash_64 and our poor multi-index hash-mixing
function, use a minutely adapted version of Austin Appleby's 32-bit
MurmurHash3 computation. It spreads keys and especially vectors of
keys much better.
Abegail Jakop [Tue, 15 Mar 2016 20:08:20 +0000 (16:08 -0400)]
PR13608: poll checking if the stap server started
systemtap.exp: in proc setup_server, instead of sleeping for 1 second
between starting the server and checking if it started, which
sometimes fails on rhel7, wait and poll whether the stap server has
started.
Abegail Jakop [Tue, 1 Mar 2016 21:14:27 +0000 (16:14 -0500)]
PR13608: --max-compressed-request ## server option
add --max-compressed-request compile server option that takes a value
for the maximum compressed client request size allowed in bytes. The
default, if this option is not used is 5000 bytes.