Frank Ch. Eigler [Wed, 23 Mar 2016 22:02:15 +0000 (18:02 -0400)]
stapio main loop: always time out on pselect()
The main thread's pselect() call must now get a timeout setting too,
because it is responsible for orderly shutdown of the stap module in
case of interrupts. Without a polling timeout, the pselect() can just
sit there, with other reader threads happily continuing their work,
as though the ^C never took effect.
David Smith [Wed, 23 Mar 2016 20:44:10 +0000 (15:44 -0500)]
Fix staprun compile error when HAVE_MONITOR_LIBS isn't defined.
* staprun/common.c (monitor_winch): Add empty function when
HAVE_MONITOR_LIBS isn't defined, to make the compiler happy. This is
necessary since the function's address is used in setup_main_signals().
* staprun/staprun.h: When HAVE_MONITOR_LIBS isn't defined, change
monitor_winch() from an inline function to a declaration.
William Cohen [Wed, 23 Mar 2016 20:10:17 +0000 (16:10 -0400)]
Avoid "make distclean" removing removing various man pages
The man pages are not generated by the regular "make". They are now
generated using a special step and are now part of the sources. The
"make distclean" should not remove these prebuilt pages.
Frank Ch. Eigler [Wed, 23 Mar 2016 14:48:28 +0000 (10:48 -0400)]
monitor mode: staprun revamp
A group of changes for usability & stability of the staprun side of
--monitor mode. The gist of it is to let staprun persist awhile after
a script exits (whether interactively commanded with 'q', or with the
script exiting due to an exit/error), so that its outputs can be seen.
* monitor.c (monitor_state): Add new "exited" and "exited_help" states.
(monitor_render): Remove code that reads input from module.
Support exited_help subset text.
(monitor_remember_output_line): New utility function to strndup
module output lines into our circular buffer.
(monitor_input): Revamp logic to use read(2) syscalls instead of
temporary fdopen() FILEs to read from module-output pipe, for
several reasons.
(monitor_exited): New function to enter exited state.
* relay.c (reader_thread): Use write(2) loop when feeding output
pipe/file, in case it gets full.
(init_relayfs): Drop F_SETPIPE_SZ; it is not needed with write loop
here and proper nonblocking read on other side.
* mainloop.c (signal_thread): Keep it running after each signal, just
in case a ^C is repeated. Encode SIGQUIT with a set to 'load_only'
instead of pending_interrupts count.
(stp_main_loop): Treat several startup-time errors as worthy of
stap module removal/cleanup rather than detaching. Within main
loop proper, call cleanup_and_exit upon pending_interrupts, as
the single standard spot where a stap module shutdown from any
cause should come. Redirect warning/error messages to monitor
output buffer. Notify monitor code of received STP_EXIT message.
* staprun.h: Provide inline-void stub functions for the !MONITOR_LIBS
case.
David Smith [Mon, 21 Mar 2016 22:08:59 +0000 (17:08 -0500)]
Fix PR17393 by improving the way we execute "system" commands.
* staprun/common.c (open_cloexec): New function.
(openat_cloexec): Ditto.
(closefrom): ditto.
* staprun/staprun.h: Add prototypes for new functions.
* staprun/ctl.c (init_ctl_channel): Before trying to open the control
channel, make sure it isn't open already. Use openat_cloexec() and
open_cloexec() when opening the control channel. Remove unneeded call to
set_clexec().
* staprun/relay.c (open_outfile): Use open_cloexec(). Remove unneeded call
to set_clexec().
(init_relayfs): Ditto.
* staprun/relay_old.c (open_relayfs_files): Ditto.
(init_oldrelayfs): Ditto.
* staprun/mainloop.c (start_cmd): Add comment explaining the closefrom()
isn't needed here.
(system_cmd): Wait until the child has closed everything (by calling
closefrom) before returning.
* staprun/staprun.c (term_signal_handler): Remove unneeded call to
close_ctl_channel(), since remove_module() now calls it.
Felix Lu [Mon, 21 Mar 2016 20:31:07 +0000 (16:31 -0400)]
Monitor mode: clamp probe list scrolling
Previously, the probe list could be scrolled beyond the number of probes
available and display an empty list. This also made it harder to scroll
up. Also, the number of script probes is now extracted using the json list.
Felix Lu [Mon, 21 Mar 2016 14:13:27 +0000 (10:13 -0400)]
Remove monitor mode global mangling
Commit 87103d2b32ea873721 does not mangle variable names
during the search so the synthetic monitor_enabled variables could
not be found. Those synthetic variables are now not mangled to
allow them to be found.
Frank Ch. Eigler [Sun, 20 Mar 2016 23:18:49 +0000 (19:18 -0400)]
monitor mode: add 'q' keystroke to quit
It'll be nice to have a sequence other than ^C to cause an exit, so
that e.g. we can review output / diagnostics after the script quit.
Switching to 'h' to flip between help screen and normal mode, to avoid
accidental 'q'.
Frank Ch. Eigler [Sun, 20 Mar 2016 22:22:01 +0000 (18:22 -0400)]
monitor mode: deconflict with private global mangling
The synthesized probe body for monitor mode control needs to refer to
globals that may be declared private inside a tapset, in order to
clear them. It needs to pierce the mangling-based name-hiding, so
that when the synthetic fragment is symbol-resolved, it will find the
mangled private globals to manipulate. This is implemented by an
optional mode-flag in the symresolution_info class to suppress
global-mangling.
Frank Ch. Eigler [Sun, 20 Mar 2016 22:20:49 +0000 (18:20 -0400)]
translator: PR13283 redux, optimize string comparison again
With the function-overloading machinery now assuring that function
calls translate to full temporaries to hold return values, we no
longer need to make yet another copy just to evaluate string
comparisons.
Felix Lu [Fri, 18 Mar 2016 18:11:05 +0000 (14:11 -0400)]
Add some monitor mode improvements.
Added an option to pause and resume the script while running to
allow the user to review the output. Previously, a synthetic probe
was being displayed in the probe list, this is now removed.
* elaborate.cxx:
- monitor_mode_write: skip generating probe point
condition for synthetic probes. Add command to resume
and pause all probes.
- semantic_pass: call monitor_mode_{read, write} in init
* staprun/monitor.c: Bind keys for pause and resume.
Frank Ch. Eigler [Fri, 18 Mar 2016 01:24:50 +0000 (21:24 -0400)]
PR19802: hashbench.stp
Rework into the beginning of a hash benchmark suite.
The first test is for plotting 1/2/3-d hash value distributions,
with a bunch of modifiable parameters (use stap "-G param=value").
Frank Ch. Eigler [Thu, 17 Mar 2016 20:19:21 +0000 (16:19 -0400)]
stap dyninst: allow -B option to pass CFLAGS
Formerly, -B (session.kbuildflags) was only used during runtime=kernel
builds. Now it can be used to pass CFLAGS to the compiler run that
builds the generated dyninst-bound stap_FOO.so shared library.
Frank Ch. Eigler [Thu, 17 Mar 2016 20:04:51 +0000 (16:04 -0400)]
dyninst runtime: make ilog2 evaluable even with -O0
While debugging a generated dyninst module, it became an obstacle that
the __builtin_constant_p-guarded ilog2() macro expression failed down
to a ____ilog2_NaN dummy function. With -O0, that function is at least
referenced, thus blocking linkage. We ditch the constant-p checking,
and provide an abort()ing fallback.
Frank Ch. Eigler [Thu, 17 Mar 2016 19:25:19 +0000 (15:25 -0400)]
PR19802: inline murmurhash3 into map-gen.h
Instead of calling into the distributed murmurhash3 function, break it
up so that the map-gen.h hash function inlines it. Each index
(integer or string) is mixed into the hash naturally as though it were
copied into one large contiguous block.
The results show very good dispersion when compared with the kernel
hash_64() function. Evaluating the hash function is somewhat slower
(30%-ish). (Dumbing it down a little bit was attempted a couple of
ways; none were superior enough in dispersion/performance.)
Frank Ch. Eigler [Mon, 14 Mar 2016 01:33:51 +0000 (21:33 -0400)]
PR19802 prototype: switch to murmurhash3
Instead of the kernel's hash_64 and our poor multi-index hash-mixing
function, use a minutely adapted version of Austin Appleby's 32-bit
MurmurHash3 computation. It spreads keys and especially vectors of
keys much better.
Abegail Jakop [Tue, 15 Mar 2016 20:08:20 +0000 (16:08 -0400)]
PR13608: poll checking if the stap server started
systemtap.exp: in proc setup_server, instead of sleeping for 1 second
between starting the server and checking if it started, which
sometimes fails on rhel7, wait and poll whether the stap server has
started.
Abegail Jakop [Tue, 1 Mar 2016 21:14:27 +0000 (16:14 -0500)]
PR13608: --max-compressed-request ## server option
add --max-compressed-request compile server option that takes a value
for the maximum compressed client request size allowed in bytes. The
default, if this option is not used is 5000 bytes.
move the compressed client request size check from check_request_size()
to readDataFromSocket() which will check the expected request size
before the request transmits. Then rename check_request_size()
to be more specific, so check_uncompressed_request_size(). Default
max compressed request size if 5000 bytes.
Abegail Jakop [Thu, 25 Feb 2016 20:28:33 +0000 (15:28 -0500)]
PR13608: add --max-request-size ## option for server
Add an option in the stap server to set the max (uncompressed)
request size in bytesreceived from the client. If it's not set,
the default max request size is 50KB. Then the max compressed
request size is set to be 10% of the max uncompressed size, making
the default 5KB.
Martin Cermak [Mon, 14 Mar 2016 07:39:22 +0000 (08:39 +0100)]
Preserve the working directory within the test.
Test must not change the working directory. Without this update,
when target_compile() failed, test finished with working directory
set to $tmpdir instead of $curdir breaking all subsequent tests:
=====
Running ./systemtap.base/utrace_syscall_args.exp ...
UNTESTED: 64_BIT_UTRACE_SYSCALL_ARGS
Running ./systemtap.base/valid_pid.exp ...
ERROR: ./systemtap.base/valid_pid.exp does not exist.
...
=====
David Smith [Fri, 11 Mar 2016 18:36:18 +0000 (12:36 -0600)]
Fixed PR19799 by making deleting from aggregate arrays more reliable.
* runtime/pmap-gen.c (_stp_pmap_del): Be sure to delete keys from all
cpu's maps.
* runtime/map-gen.c (_stp_map_del_hash): New function.
* testsuite/systemtap.maps/agg_delete.exp: New test case.
* testsuite/systemtap.maps/agg_delete1.stp: Ditto.
* testsuite/systemtap.maps/agg_delete2.stp: Ditto.
Felix Lu [Wed, 9 Mar 2016 14:22:40 +0000 (09:22 -0500)]
Use function overloading in some tapset functions
Documentation for overloaded functions are now merged into
one entry. Some tapset functions are now simplified through
overloading. Instead of having new function names with suffixes
such as "2" or "pid" to indicate extra arguments, the functions
now seem to have optional arguments.
* doc/SystemTap_Tapset_Reference/overload.py: New script to resolve
documentation for overloaded functions.
* tapset/*: Rename some functions.
* testsuite/*: Rename functions in tests.
Remove call to create_tmp_dir() from the main systemtap_session
constructor and call it slightly later, after the cmdline parsing
is finished. Plus modify create_tmp_dir() so that it only creates
the tmpdir if --tmpdir has not been set on the cmdline.
David Smith [Fri, 4 Mar 2016 15:42:34 +0000 (09:42 -0600)]
Make "monitor" mode working with "interactive" mode.
* interactive.cxx (forked_parse_pass): Make sure "monitor" mode is off
while running the parser. Otherwise we can end up with multiple copies
of the monitor synthetic probes.
(forked_semantic_pass): Ditto.
(interactive_mode): Make sure "monitor" mode is off while dumping probe
types.
David Smith [Thu, 3 Mar 2016 19:58:56 +0000 (13:58 -0600)]
Make utrace_syscall_args.exp pass when run on an NFS filesystem.
* testsuite/systemtap.base/utrace_syscall_args.exp: Put all temporary
files in a temporary directory and run the script from there. This makes
the test pass when run from an NFS filesystem.
David Smith [Thu, 3 Mar 2016 19:56:39 +0000 (13:56 -0600)]
Make sure the "dump" options can't be used with -i or --monitor.
* session.cxx (check_options): Make sure the dump options (-l/-L/--dump-*)
can't be used with interactive mode (-i) or monitor mode (--monitor).
* testsuite/parseko/cmdline25.stp: New test.
* testsuite/parseko/cmdline26.stp: Ditto.
David Smith [Wed, 2 Mar 2016 19:27:53 +0000 (13:27 -0600)]
In interactive mode, run the user's script through -p2 after each change.
* interactive.cxx (auto_analyze_opt): Added new class to handle the
"auto_analyze" option. If non-zero, the user's script is run through
pass 2 after every command that modifies the script.
(add_cmd::handler): Added logic to call forked_semantic_pass() if the
auto_analyze option is set.
(edit_cmd::handler): Ditto.
(delete_cmd::handler): Ditto.
(forked_semantic_pass): New function to run the user's script through
pass 2.
(interactive_mode): Add new "auto_analyze" option.
Felix Lu [Tue, 23 Feb 2016 20:38:00 +0000 (15:38 -0500)]
PR18431: Function overloading
A functioncall may now have a variety of implementations
and have the selection happen during runtime. Overloading
by number of parameters during compile time is also
available.
* elaborate.cxx:
- no_var_mutation_during_iteration_check::visit_functioncall:
Adapt to referents list.
- symresolution_info::visit_functioncall: Likewise + sort functioncalls.
- symresolution_info::find_function -> find_functions: Likewise.
- void_statement_reducer::visit_functioncall: Likewise.
- duplicate_function_remover::visit_functioncall: Likewise.
- autocast_expanding_visitor::resolve_functioncall: Likewise.
- typeresolution_info::visit_arrindex: Likewise.
- get_functionsig: Use unmangled name.
* elaborate.h: find_function -> find_functions.
* main.cxx: Set parser user_file indicator flag.
* parse.cxx: Parser user_file flag, accept function declaration
priority label syntax, remove duplicate function name checking,
allow next in functions.
* parse.h: pf_user_file flag.
* common_probe_context.h: New "next" flag.
* session.h: New overload_count map for overload name resolution.
* staptree.cxx:
- functioncall::functioncall: Remove referent initializer list.
- functiondecl::functiondecl: Initialize default function priority.
- functioncall_traversing_visitor::visit_functioncall:
Adapt to referents list.
- deep_copy_visitor::visit_functioncall: Likewise.
* staptree.h:
- functioncall: Now holds list of referents.
- functiondecl: New priority value.
- function_priority_order: New functor for sorting.
* tapset-method.cxx: Adapt to referents list.
* tapsets.cxx: Adapt to referents list.
* parseko/eleven.stp: Invalidated.
* parseko/nineteen.stp: Invalidated.
* parseko/private02.stp: Invalidated.
* semko/fifty.stpL Invalidated.
* systemtap.base/optim_arridx.exp: Make compatible with overloading.
* translate.cxx:
- c_unparser: New fc_counter to generate functioncall end labels.
- c_unparser::emit_function: Reset context next flag, output STAP_NEXT
macro.
- c_unparser::visit_next_statement: Set context next flag, allow next
in functions.
- c_unparser::visit_functioncall: Main overloading logic.
- recursion_info::visit_functioncall: Adapt to referents list.
David Smith [Tue, 1 Mar 2016 19:34:59 +0000 (13:34 -0600)]
Improve the interactive mode "add" command by prompting for more input.
* interactive.cxx (add_cmd::handler): If the user doesn't enter a complete
function or probe definition on the command line, keep prompting him to
do so. We know he's finished when the definition can be parsed without
an error.
(forked_parse_pass): New function.
* tapset/print_stats.stpm: Add prints[1-9] macros.
* parse.cxx (parser::slurp_pp1_param): Also track [ ] nesting
when slurping macro parameters, as commas might appear in there
that need to be relayed rather than interpreted as macro-param
separators.
* testsuite/systemtap.printf/printf_stats.*: New test.
David Smith [Fri, 26 Feb 2016 19:32:27 +0000 (13:32 -0600)]
Add multiarch testing to testsuite/systemtap.base/callee.exp.
* testsuite/systemtap.base/callee.exp: Revamp test case to test all
supported executable sizes on multiarch systems. Standardize on a
4-character indent.
David Smith [Wed, 24 Feb 2016 22:10:16 +0000 (16:10 -0600)]
Fix the threadstacks.stp example and testcase for ppc64.
* testsuite/systemtap.examples/process/threadstacks.stp: Add ppc64 support
by also probing "/lib*/*/libpthread.so.*", since RHEL7 ppc64 puts the
pthread library in /lib64/power8/.
* testsuite/systemtap.base/pthread_stacks.exp: Choose the new stack size
based on architecture, since 0x10000 is invalid for ppc64.
David Smith [Wed, 24 Feb 2016 19:02:55 +0000 (13:02 -0600)]
Update 2 test cases to work on ppc64, which can inline vfs_{read,write}.
* testsuite/systemtap.base/onoffprobe.stp: Switch from probing the return
of vfs_write() (which can be inlined) to probing the return of
syscall.write.
* testsuite/systemtap.base/listing_mode_sanity.exp: Make sure we use
'.call' when listing kernel.function("vfs_read"), since on some kernels
that function can be inlined.
David Smith [Wed, 24 Feb 2016 17:14:41 +0000 (11:14 -0600)]
Get more passes in semok.exp and semko.exp on ppc64.
* dwflpp.cxx (build_kernel_blacklist): Add ppc64-specific
include/asm/io-defs.h file to the blacklist.
* testsuite/systemtap.pass1-4/semko.exp: Make sure vfs_read() doesn't have
an inlined version before running twentytwo.stp.
* testsuite/systemtap.pass1-4/semok.exp (dyninst_kfails): Check for plt
probe support before running plt[12].stp.
David Smith [Tue, 23 Feb 2016 19:23:14 +0000 (13:23 -0600)]
Fix rawhide compile problems in aux_syscalls.stp.
* tapset/linux/aux_syscalls.stp (_stp_siginfo_u): Removed extra "break"
statements that gcc version 6 complains about on rawhide.
(_stp_compat_siginfo_u): Ditto.
William Cohen [Thu, 18 Feb 2016 22:08:12 +0000 (17:08 -0500)]
Accept other error codes returned by socket syscall in socket.c syscall test
On aarch64 the last socket syscall in socket.c return EINVAL rather
than EPROTONOSUPPORT. Adjust the test to be a bit more flexible. An
error is an error.