Jonathan Lebon [Mon, 2 Jun 2014 21:22:52 +0000 (17:22 -0400)]
hrtimer probes: support on-the-fly [dis]arming
This patch adds support for on-the-fly arming/disarming of hrtimer
probes. Although it is only supported for non-usermode, there were a few
changes required in dyninst/timer.c so that the same interface is
exposed for both usermode and kernel mode.
runtime/dyninst/timer.c
- Decouple timer creation from timer starting by factoring out
_stp_hrtimer_start() from _stp_hrtimer_create().
- Make _stp_hrtimer_cancel() actually just cancel the timer, and not
completely delete it.
- Add new _stp_hrtimer_delete() to delete the timer.
runtime/linux/timer.c
- Similarly, factor out _stp_hrtimer_start() from
_stp_hrtimer_create().
- Add new _stp_hrtimer_delete(), which also does a cancel.
tapset-timers.cxx
- Declare hrtimer_derived_probe as a probe that supports on-the-fly
operations.
- In emit_module_init(): unconditionally create the timer, but if in
STP_ON_THE_FLY mode, only bother to start it if its condition is
met.
- In emit_module_refresh(): check for which timers to start/cancel.
Jonathan Lebon [Fri, 6 Jun 2014 17:52:39 +0000 (13:52 -0400)]
runtime: lock module_refresh with mutex
If STP_ON_THE_FLY is enabled, then we need to ensure that
systemtap_module_refresh() is never run concurrently. Since we use a
workqueue, no concurrency can occur from arm/disarm refreshes alone.
However, the module notifier could fire at any time.
Jonathan Lebon [Mon, 2 Jun 2014 15:51:38 +0000 (11:51 -0400)]
runtime: use workqueue to schedule systemtap_module_refresh()
In this patch, we check in the probe handler common epilogue whether a
module refresh is needed as determined by the probe handlers. If so, we
schedule_work() so that systemtap_module_refresh() is eventually called
to update the necessary probes.
Jonathan Lebon [Mon, 2 Jun 2014 15:22:17 +0000 (11:22 -0400)]
translate.cxx: evaluate conditions in module_init and handlers
This patch ensures that the stap_probes[] cond_enabled field is updated
as required: once during systemtap_module_init(), and as needed in probe
handlers that may affect the result of condition evaluation (using the
derived_probe::affected_probe set populated in
semantic_pass_conditions()).
If the new evaluated value of the condition is different from its
previous value, then need_module_refresh is set (if this type of probe
supports on-the-fly arming/disarming), which will be acted upon in a
future patch.
Jonathan Lebon [Mon, 2 Jun 2014 19:15:51 +0000 (15:15 -0400)]
derived_probe: add on_the_fly_supported()
We add a new function derived_probe::on_the_fly_supported(), which
defaults to false. As we add on-the-fly arming/disarming to various
probe types, we simply need to override this function to notify the
runtime that the operation is supported (and thus e.g. to refresh the
module when necessary).
In semantic_pass_conditions(), we now only turn on STP_ON_THE_FLY if one
of the probes with a condition supports it.
Jonathan Lebon [Fri, 6 Jun 2014 17:50:39 +0000 (13:50 -0400)]
runtime: split stap_probe struct definition from stap_probes
Because probe handlers will need to refer to the stap_probes[]
cond_enabled field, we need to define the stap_probe struct prior to the
probe handlers.
This patch achieves this by moving probe emitting in between the
stap_probe struct definition and the actual stap_probes[] array
declaration.
Jonathan Lebon [Fri, 6 Jun 2014 17:56:36 +0000 (13:56 -0400)]
stap_probe struct: add cond_enabled field
The new cond_enabled field represents whether the condition of the
associated probe handler currently evaluates to true or false. This
field will be re-evaluated in probe handlers. This is why we also need
to unconst the stap_probes[] array.
In anticipation for on-the-fly probe arming/disarming, we change
semantic_pass_conditions() to not only inline the probe condition into
its body, but also to collect, for each probe, the set of probes whose
conditions may change after the probe's handler is run. That set is
stored in the new derived_probe::probes_with_affected_conditions.
These sets will be used by the translator to emit code that will check
whether affected probes should be armed/disarmed after a handler is run.
We also introduce the STP_ON_THE_FLY define which will gate all
on-the-fly related code and will be emitted only if required. Finally,
we have STP_ON_THE_FLY_DISABLED which can be used to disable all
on-the-fly arming/disarming.
Jonathan Lebon [Wed, 21 May 2014 18:51:34 +0000 (14:51 -0400)]
java probes: remove probe group
Java probes decay to SDT marker probes. There is no Java-specific code
that needs to be emitted during C unparsing and thus no
java_derived_probe_group needed. We remove forward decls to a
non-existent java_derived_probe_group struct and the
java_derived_probes reference in systemtap_session.
Frank Ch. Eigler [Sun, 10 Aug 2014 01:16:32 +0000 (21:16 -0400)]
PR17249: tolerate early module notifier calls with null mod->sect_attrs
In the case of MODULE_STATE_COMING, we may encounter NULL sect_attrs,
and we must not crash. Sadly, that case can mean the loss of ability
to probe module-init functions - i.e., breaking the bz6503 test case.
* runtime/transport/symbols.c (_stp_module_notifier): Don't assume
that mod->sect_attrs is valid. Treat COMING|LIVE notifications
similarly, except LIVE should assume init.* gone gone gone,
she been gone so long, she been gone gone gone so long.
Frank Ch. Eigler [Sun, 10 Aug 2014 01:31:34 +0000 (21:31 -0400)]
PR17232 take #3: mutex the control messages
As per jistone's advice, simplify control message control by imposing
a mutex over the whole receive-side handling of a ctl message. That
precludes concurrent or reentrant messages (independent of /ctl
open-time limits or threading assumptions). It lets the start and
exit handling functions keep track with fewer state variables.
In a way, this elaborates upon a reversion of commit #262f7598.
* runtime/transport/control.c (_stp_ctl_write_cmd): Use a new static
cmd_mutex for ctl message handling. Don't bother with counters and
flags for startedness etc; let the lower level functions handle
that. Handle error exits via goto out instead of return to assure
mutex unlocks.
* runtime/transport/transport.c (_stp_handle_start,
_stp_cleanup_and_exit): Drop the stp_transport_mutex control,
explain why unnecessary. Be more paranoid during module-notifier
cleanup.
PR17232 variant #2: in runtime, let STP_EXIT nest within STP_START
Uncommitted variant #1 consisted of using module refcounts in the
generated systemtap_module_init/exit function pair to ensure that an
uncleaned module cannot be unloaded. That precluded cleanup via
rmmod(8), so a robust but inconvenient solution.
This variant #2 consists of a surgical fix, wherein an STP_EXIT
message comes in during an STP_START is used to set an atomic flag for
deferred _stp_cleanup_and_exit() handling.
A variant #3 is coming soon, using a protocol-wide command-message
mutex, like we did back before commit #262f7598.
Josh Stone [Fri, 8 Aug 2014 20:46:56 +0000 (13:46 -0700)]
PR17242: Initialize tapset global arrays early on
Some of the tapset arrays were lazy-initialized on their function's
first call, but that requires the caller to always take a write lock,
and also makes those functions impure.
Now there is a "probe init" alias for the earliest possible begin probe,
and all of these arrays use that alias to initialize their contents.
None of these are big enough to expect noticeable overhead from having
to always initialize them.
Stan Cox [Fri, 8 Aug 2014 18:31:23 +0000 (14:31 -0400)]
Keep perf counters in a vector instead of an unordered map.
* session.h (perf_counters): Use a vector instead of unordered map so
iteration is in a predictable order.
* tapsets.cxx (visit_perf_op): Use it.
(dwarf_derived_probe): Likewise
(emit_probe_local_init): Likewise
(emit_module_utrace_decls): Likewise
(emit_module_inode_decls): Likewise
* tapset-perfmon.cxx (perf_builder::build): Likewise
* elaborate.h (perf_counter_refs): Make a set of strings.
* perf.sh: New test.
* perf.exp: Use it.
David Smith [Fri, 1 Aug 2014 15:53:18 +0000 (10:53 -0500)]
Add scripts to test systemtap probes on a set of kernel functions.
* scripts/kprobes_test/stap_probes_test.py: New script. While debugging
PR17140, I needed a script to bisect a list of kernel functions to put
systemtap probes on.
* scripts/kprobes_test/stap_gen_code.py: Ditto.
* scripts/kprobes_test/stap_run_module.py: Ditto.
* scripts/kprobes_test/README: Update with better instructions.
* scripts/kprobes_test/config_opts.py: Update comments.
* scripts/kprobes_test/monitor_system.py: Add beaker system monitoring
instructions.
cmdline: added "E" to the short options
elaborate: adapted to be able to perform the semantic pass on the
multiple inputs from user_files
main: parse and store the inputs in user_files, and give the additional
scripts new source "file" names (<input##>).
parse: parse function now has a parameter for the source "file" name
session: accept and store additional scripts for parsing, separate from
the cmdline_script. an additional script won't set have_script to true.
Added a few more unexplained backtraces (as visible with -Gdebug=1
runs), automated explanation-width calculations, and compressed the
initialization of the explanation/priority lookup tables.
Jonathan Lebon [Mon, 21 Jul 2014 14:58:27 +0000 (10:58 -0400)]
initscript: check final initramfs for systemtap files
As a sanity-check for a successful initramfs creation, we check using
lsinitrd that if a script was inserted then the initramfs contains the
staprun executable. This would indicate that dracut properly picked up
the stap dracut module and included it in the initramfs.
David Smith [Fri, 18 Jul 2014 20:49:39 +0000 (15:49 -0500)]
Fixed PR17181 by making utrace handle interrupting processes better.
* runtime/stp_utrace.c (utrace_init): Use kallsyms_lookup_name() to lookup
"wake_up_state" if needed.
(stp_task_notify_resume): New function to handle the details of adding a
resume handler.
(utrace_cleanup): Only output debug printk's if STP_TF_DEBUG is defined.
(utrace_set_events): Improve check.
(utrace_do_stop): Call stp_task_notify_resume() instead of inline code.
(utrace_wakeup): Call stp_wake_up_state() instead of wake_up_process()
to avoid a WARN(). Call stp_task_notify_resume() instead of inline
code.
(utrace_control): Call stp_task_notify_resume() instead of inline code.
(finish_report): Ditto.
(finish_resume_report): Add UTRACE_INTERRUPT support.
(utrace_resume): Handle UTRACE_INTERRUPT.
* runtime/linux/task_finder2.c (stap_task_finder_post_init): Go back to
sending UTRACE_INTERRUPT to all tasks.
* buildrun.cxx (compile_pass): Add export tests for 'wake_up_state' and
'try_to_wake_up'.
* runtime/linux/runtime.h: Added 'kallsyms_wake_up_state' declaration when
necessary.
* testsuite/systemtap.base/process_resume.c: New file.
* testsuite/systemtap.base/process_resume.exp: New file.
Jonathan Lebon [Thu, 10 Jul 2014 21:08:23 +0000 (17:08 -0400)]
modules_out_of_tree.exp: adapt for RHEL5
We put the opening braces on the same line as the function declaration
so that the reported lineno in listing mode is the same on RHEL5 GCC
(which considered the opening brace as the decl_line).
We also skip the probing part by checking for module refresh support by
using the new module_refresh_p proc.
Jonathan Lebon [Wed, 9 Jul 2014 14:44:43 +0000 (10:44 -0400)]
PR17073: module probes: support out-of-tree modules
This patch adds support for out-of-tree modules by specifying the
fullpath. This did not work before because both the _stp_module struct
and the stap_dwarf_probe struct were using the full path as the module
name. This caused issues during relocation of module sections/kprobe
addresses.
In this patch, we use the already existing modname_from_path() function
to modify the module member of the dwarf_derived_probe (which later
affects the stap_dwarf_probe struct) and to emit the proper name in
dump_unwindsym_cxt() for the _stp_module struct.
Jonathan Lebon [Wed, 9 Jul 2014 16:01:36 +0000 (12:01 -0400)]
module probes: allow full path to in-tree modules
If the user provided a full path to an in-tree module, then we don't
want to modify the path in any way. The '-' to '_' will be done later on
as needed.
David Smith [Thu, 10 Jul 2014 14:23:12 +0000 (09:23 -0500)]
Update fix for PR17127 by correctly handling TASK_INTERRUPTIBLE flag.
* runtime/linux/task_finder2.c (stap_task_finder_post_init): The 'state'
field of the task structure contains a bitfield, not a singular
value. Properly test for TASK_INTERRUPTIBLE.
listing_mode.c: add in an infinite loop for generating probe points
to a process running this program over the course of the entire
listing_mode test.
listing_mode.exp: edited portions dependent on the line numbers in the
source code. added tests for the new process(PID).* probe points.
David Smith [Wed, 9 Jul 2014 15:47:24 +0000 (10:47 -0500)]
Clean up sdt_misc.exp a bit.
* testsuite/systemtap.base/sdt_misc.exp: Make log output match the
commands being run. Catch errors on "system" calls to avoid the test
aborting with a tcl error. Do an initial cleanup to avoid test
leftovers, which might have been created by another user.
brendang advised his system lacked /usr/bin/msgfmt, which blew up po/
builds. We now detect & reject this regrettable state of existence at
configure time.
Jonathan Lebon [Mon, 7 Jul 2014 18:59:56 +0000 (14:59 -0400)]
listing_mode.exp: use stap instead of nm to get addrs
We previously relied on nm to retrieve the addresses of functions in
the executable and library. This causes issues on ppc64, in which nm
simply returns the function descriptor rather than the actual text
address.
We now instead rely on stap itself to retrieve function addresses. This
should fix all of the FAILs of this testcase on the ppc64 platform.
After collapsing the process., process(STRING), process(NUM) cases
into a single conditional, we must remember to retest the STRING
case so as to extract the parameter.
elaborate*: new function derived_probe_builder::has_param()
tapset*: added a check to make sure the PID in process(PID).*
probes correspond to a running process.