PR13338: prevent infinite recursion during process glob expansion
Previously, it was possible for a process("/usr/bin/*") probe to
expand to a list that included process("/usr/bin/[") (or indeed
some other wildcard-containing filename), which would recursively
trigger glob expansion. Now we \-quote glob results, to prevent
this, and later \-unquote to undo the damage.
* tapsets.cxx (dwarf_builder::build): quote / unquote process globs.
* testsuite/systemtap.base/bz13338.exp: New test.
* util.cxx (*_glob_chars): New/revised functions for \-quoting
some glob metacharacters.
* util.h: Corresponding changes.
* staptree.h: Crucial whitespace change.
Mark Wielaard [Tue, 1 Nov 2011 16:32:23 +0000 (17:32 +0100)]
Be explicit about SP from CFA and PC from RA dwarf unwind rules.
There are subtle differences between architectures whether or not they
should unwind the stack pointer from the call frame address and the
instruction pointer from the return address given by the dwarf call
frame instructions. Make it so each arch explicitly encodes how it
does the unwinding of these two special registers.
Mark Wielaard [Mon, 31 Oct 2011 13:29:46 +0000 (14:29 +0100)]
Make --with-elfutils work with either a source release or git checkout.
Add configure check to see if this is a source release, or a source
checkout. We need --enable-maintainer-mode if it isn't a source release
to make sure some additional files are generated. This makes working
against non-released elfutils git checkout a lot easier.
Mark Wielaard [Mon, 31 Oct 2011 09:39:10 +0000 (10:39 +0100)]
Introduce and use STP_ALLOC_SLEEP_FLAGS for use with gfp alloc functions.
When in a context where we can wait we like to use normal GFP_KERNEL.
But this might trigger the oom-killer, which might kill out own stapio
process. To suppress this use __GFP_NORETRY. All _stp_[kmz]alloc_gfp
functions that are used in user context should use STP_ALLOC_SLEEP_FLAGS
if appropriate.
Note that the kernel itself may still use GFP_KERNEL. For example
__alloc_percpu() will always use GFP_KERNEL.
Josh Stone [Fri, 28 Oct 2011 22:42:02 +0000 (15:42 -0700)]
Add the pattern for label probes in libraries
* tapsets.cxx (dwarf_derived_probe::register_patterns): Refactor so that
all process[.library] variants get the same set of suffixes.
(dwarf_derived_probe::register_function_and_statement_variants): Add
the label pattern from here, just as .inline is done.
Josh Stone [Fri, 28 Oct 2011 22:11:17 +0000 (15:11 -0700)]
Refactor sdt & plt pattern registration
This factors the sdt and plt suffixes away from the various process and
process.library prefixes. As a result, we also gain process.plt for
probing -c commands.
* tapsets.cxx (dwarf_derived_probe::register_sdt_variants): New.
(dwarf_derived_probe::register_plt_variants): New.
(dwarf_derived_probe::register_patterns): Register sdt and plt for all
process/library combinations.
* testsuite/semok/plt2.stp: Test that process.plt succeeds with -c
* testsuite/semko/plt2.stp: Test that process.plt fails without -c
Mark Wielaard [Fri, 28 Oct 2011 21:47:53 +0000 (23:47 +0200)]
Just test up to -p2 for kernel-debuginfo smoke test.
We already do a -p4 kernel-devel smoke test just before this one.
We don't need another one. It slows down doing make installcheck
RUNTESTFLAGS=quick_test.exp runs. Especially on slow machines.
Josh Stone [Fri, 28 Oct 2011 21:42:23 +0000 (14:42 -0700)]
plt: Throw an appropriate error for missing $var
* tapsets.cxx (plt_expanding_visitor::visit_target_symbol): Throw a
semantic_error for missing $var, so it may be caught and chained for
later. This ensures that the visitor always has a provide() too.
* testsuite/semko/plt1.stp: New, check that missing $var is an error.
* testsuite/semok/plt1.stp: New, check that $$name works, as does
@defined on a missing $var.
Based on peer feedback:
- Compare privilege credentials as sets, not levels.
- Send privilege credentials as a ctl message at startup.
- Send startup ctl messages (tzinfo, relocations, privilege_credentials)
on the same control channel. Ensure channel is closed properly on error.
David Smith [Thu, 27 Oct 2011 16:06:14 +0000 (11:06 -0500)]
More PR13083 fixes to avoid sleeping in the task_finder (ported).
* runtime/task_finder2.c (__stp_get_mm_path): Call down_read_trylock()
instead of down_read() to avoid the possibility of sleeping.
(stap_start_task_finder): Call task_lock() to make sure a task's mm
doesn't go away while we're looking at it.
David Smith [Thu, 27 Oct 2011 15:24:45 +0000 (10:24 -0500)]
More PR13083 fixes to avoid sleeping in the task_finder.
* runtime/task_finder.c (__stp_get_mm_path): Call down_read_trylock()
instead of down_read() to avoid the possibility of sleeping.
(stap_start_task_finder): Call task_lock() to make sure a task's mm
doesn't go away while we're looking at it.
Based on peer feedback:
- Compare privilege credentials as sets, not levels.
- Send privilege credentials as a ctl message at startup.
- Send startup ctl messages (tzinfo, relocations, privilege_credentials)
on the same control channel. Ensure channel is closed properly on error.
David Smith [Mon, 24 Oct 2011 16:37:50 +0000 (11:37 -0500)]
Fixed PR13083 by no longer calling mmput() when possible (ported).
* runtime/task_finder2.c (__stp_utrace_attach): Don't call
get_task_mm()/mmput(). Since we're in the context of that task, the mm
should stick around (and mmput() can sleep).
(__stp_call_mmap_callbacks_with_addr): Ditto.
(__stp_utrace_attach_match_tsk): Ditto.
(__stp_call_mmap_callbacks_for_task): Ditto.
Mark Wielaard [Fri, 21 Oct 2011 14:49:02 +0000 (16:49 +0200)]
Extract addr_of for unprivileged_[my]probes.exp once and with optional dot.
Some architectures prefix function symbols with a '.', so accept an
optional dot prefix. Also extract each symbol only once instead of
for each pass over the list.
Mark Wielaard [Fri, 21 Oct 2011 11:34:21 +0000 (13:34 +0200)]
Extend vma_map entry vm_end range when additional segments are mapped.
Once registered, we may want to extend an earlier registered region.
A segment might be mapped with different flags for different offsets.
If so we want to record the extended range so we can address more
precisely to module names and symbols.
David Smith [Wed, 19 Oct 2011 15:00:45 +0000 (10:00 -0500)]
Fixed PR13158 by adding a staprun signal handler so modules don't remain.
* runtime/staprun/staprun.c (term_signal_handler): New function.
(setup_term_signals): Ditto.
(insert_stap_module): Remembers if insert_module() worked.
(main): Call setup_term_signals().
* testsuite/systemtap.base/rename_module.exp: Made test case more robust
by waiting on output from all stap scripts.
* testsuite/systemtap.base/pr13158.exp: New testcase.
* transport/relay_v2.c, relayfs.c (_stp_transport_data_fs_init):
Increment _stp_allocated_[net_]memory with relay buffers.
* transport/ring_buffer.c: Ditto. Plus fix scaling of the incoming -sNN
value, so NN represents megabytes consistently.
Frank Ch. Eigler [Tue, 18 Oct 2011 20:58:15 +0000 (16:58 -0400)]
PR13317: improve parse error recovery heuristics
This change changes the 10 parse errors reported by brendang for iostat-scsi.stp
to just six actually salient ones.
* parse.cxx (parser::parse): Reorder error recovery to skip
everything until next top-level keyword. Wrap try {} in
while {} rather than vice versa.
(parser::parse_stmt_block): Drop recovery loop.
Dave Brolley [Tue, 18 Oct 2011 17:54:20 +0000 (13:54 -0400)]
PR 13128: Pass user privilege credentials from staprun to the module.
- User's privilege credentials are determined by staprun and passed to the module
- Module contains generated code which checks the user's credentials against those
required to load/run the module (specified at compile time).
- module aborts if credentials are insufficient.
Mark Wielaard [Tue, 18 Oct 2011 12:28:59 +0000 (14:28 +0200)]
PR13284 Failure to resolve kernel global e.g. jiffies.
* dwflpp.cxx (vardie_from_symtable): Also accept STT_NOTYPE. Some global
don't have a type associated with them.
* testsuite/buildok/pr13284.stp: New testcase.
Josh Stone [Mon, 17 Oct 2011 22:18:06 +0000 (15:18 -0700)]
PR13283: Use a temporary string for comparisons
The reported issue was trying to directly compare the results of two
different kernel_string() calls, but they were always computed equal
even though they were known to be different. Inspection of pass-3
showed that both calls led to an expr-stmt ending in ...->__retvalue.
This is just a pointer though, and it happens here that both calls will
point to the same string buffer, so strncmp rightly calls them equal.
To fix this, we must introduce temporary storage for the intermediate
strings. Now we won't lose either value, and they can be properly
compared for equality.
* translate.cxx (c_tmpcounter::visit_comparison): Declare temporary
storage for the left operand if it's not a literal.
(c_unparser::visit_comparison): Use the left temporary.
* testsuite/systemtap.string/strcmp.*: Test string comparisons.
Josh Stone [Thu, 13 Oct 2011 22:55:24 +0000 (15:55 -0700)]
Tweak cache logic to group removals by hash value
We used to glob the cache for *.ko, *.so, *.h, and *.log, implicitly
assuming that this was all that was cached. For the .ko case we also
grouped .c and .sgn to be removed all together. But when any new
extension came along, the cache cleanup ignored it.
Now we glob instead for any file in the cache that looks like is has a
hashed name, and we group removals for all files that appear to have the
same hash value, regardless of prefix or suffix. This will hopefully
make us more robust against future file-caching changes.
* cache.cxx (cache_ent_info): Becomes a vector of paths, with no
knowledge needed for modules. Use better types for size and mtime.
(clean_cache): Use a new glob that looks for hash-like files, with 32
hex digits in a row. Use a regex to find common HASH_LEN substrings
in all those filenames. Build the cache_ent_info groups by HASH_LEN,
and proceed to delete old entries until below quota.
Mark Wielaard [Thu, 13 Oct 2011 21:19:21 +0000 (23:19 +0200)]
Handle PPC64 function symbols by resolving function descriptors.
* translate.cxx (dump_symbol_table): If the module is EM_PPC64, but not
ET_REL, then register two address for a STT_FUNC. The function descriptor
address and the address of the function that the descriptor points to.
For ET_REL still rely on the .function_name convention.
David Smith [Thu, 13 Oct 2011 21:24:43 +0000 (16:24 -0500)]
(More PR13289 fixes) Added better %m/%M error handling.
* runtime/print.c (_stp_unreserve_bytes): New function.
* runtime/print.h: Added _stp_unreserve_bytes() declaration.
* runtime/vsprintf.c (_stp_vsprint_memory): Only change the input pointer
to "<NULL>" when we're formatting a string (%s), not when we're
formatting raw memory (%m/%M). Also, return NULL on errors.
(_stp_vsprint_memory_size): Only change the input pointer to "<NULL>"
when we're formatting a string (%s), not when we're formatting raw
memory (%m/%M).
(_stp_vsnprintf): If _stp_vsprint_memory() returns NULL, unreserve bytes
in the print buffer, when necessary.
* translate.cxx (c_unparser::emit_compiled_printfs): If
_stp_vsprint_memory() returns NULL, unreserve bytes in the print buffer,
when necessary. Also improve error message if _stp_vsprint_memory()
fails by setting 'last_stmt'.
* testsuite/systemtap.stress/conversions.stp: Added tests for '%M' and for
using sprintf() with '%m' and '%M'.
* testsuite/systemtap.stress/conversions.exp: Updated error count.
Josh Stone [Thu, 13 Oct 2011 20:22:29 +0000 (13:22 -0700)]
PR13155: Fix tracepoint argument access
The restructured tracepoint build broke the way tracepoint parameters
are passed to the script. This patch cleans that up, and simplifies a
few other parts of the code generation.
* tapsets.cxx (tracepoint_derived_probe_group::emit_module_decls):
Define intptr_t for the aux source. Collect .used entries from
p->args[] into used_args[] to ease call-list generation. Simplify
registration by replacing the wrappers with plain declarations.
* buildrun.cxx (compile_pass): We do have everything needed for strict
prototypes, so don't squash this warning.
Frank Ch. Eigler [Thu, 13 Oct 2011 16:33:48 +0000 (12:33 -0400)]
PR13155: separate compilation for tracepoint-related functions
The linux kernel's tracepoint headers are sometimes mutually
conflicting, defining structs etc. differently. Since we need to use
each header in its natural habitat, we need to separately-compile
those tracepoint-related functions that absolutely need access to
those declarations. This is quite new to us as so far, the translator
has usually emitted a single translated_source output, plus the
stap-symbols.h file.
* buildrun.cxx (compile_pass): Compile auxiliary sources into main module.
* tapsets.cxx (tracepoint_derived_probe_group::emit_module_decls): Effectively
rewrite to create per-tracepoint-header auxiliary source files.
* session.cxx (op_create_auxiliary): New function.
* translate.cxx (translator_output): Add filename field.
* session.h, translate.h: Corresponding changes.
* hash.cxx (find_script_hash): Rename primary source file to stap_XXX_src.c.
* main.cxx (passes_0_4): Ditto.
David Smith [Thu, 13 Oct 2011 17:37:29 +0000 (12:37 -0500)]
Fixed PR13083 by no longer calling mmput() when possible.
* runtime/task_finder.c (__stp_utrace_attach): Don't call
get_task_mm()/mmput(). Since we're in the context of that task, the mm
should stick around (and mmput() can sleep).
(__stp_call_mmap_callbacks_with_addr): Ditto.
(__stp_utrace_attach_match_tsk): Ditto.
(__stp_call_mmap_callbacks_for_task): Ditto.
Chris Meek [Thu, 13 Oct 2011 17:04:54 +0000 (13:04 -0400)]
PR13261: Added map->wrap mode to script parser
To activate map->wrap mode, use the '%' symbol after the variable name
in a global declaration, for example:
global foo%[100]
This will let it automatically replace elements in the map if you
try to place something out of bounds, rather than give an error.
It also works for statistics-valued arrays.
parse.cxx
Checked for the '%' symbol in parser::parse_global(), and if
present, sets the vardecl* d->wrap value to true.
staptree.cxx
Set the default value of the new vardecl->wrap boolean value to false.
Added the '%' to the hash if it is enabled.
staptree.h
Added the bool wrap element to struct vardecl
translate.cxx
Added the bool wrap element to struct mapvar and initialixed it to the
wrap element of the associated vardecl variable.
Checked whether the mapvar->wrap has been set (from vardecl->wrap set
in parser::parse_global()).
If so, added the apropriate code to set the map->wrap for whether its a
map or a pmap.
Mark Wielaard [Thu, 13 Oct 2011 11:53:08 +0000 (13:53 +0200)]
PR13241 too much dump-unwindsyms processing for modules even if !need_symbols
Introduce dump_section_list() that will pregenerate the section list when
needed. dump_symbol_tables() now does the minimum necessary to extract
the needed symbols, or will not even be called at all when !need_symbols
and the module isn't the "kernel".
David Smith [Wed, 12 Oct 2011 21:52:30 +0000 (16:52 -0500)]
Fixed PR12389 by making %m/%M use kread() to access memory.
* runtime/vsprintf.c (_stp_vsprint_memory): The '%m' and '%M' formatting
operators use kread() to read memory.
* translate.cxx: Removed all support from the c_unparser class for
'probe_or_function_needs_deref_fault_handler' processing since it is no
longer needed.
(c_unparser::visit_print_format): Removed deref_buffer() call. Added
check for c->last_error after print is done.
* runtime/runtime_context.h: New file.
* runtime/vsprintf.h: Ditto.
* runtime/print.c: Removed extra include of vsprintf.c. Allow use to
override STP_LOG_BUF_LEN. Make _stp_print_flush() definitions exactly
the same between the RELAY_{GUEST,HOST} and normal caes.
* runtime/print.h: Added a few more functions and includes the new
vsprintf.h file.
* runtime/runtime.h: Includes print.h instead of print.c. Moved print.c
and io.c to runtime_context.h.
* runtime/transport/transport.h: Declares '_stp_target'.
Mark Wielaard [Wed, 12 Oct 2011 10:53:30 +0000 (12:53 +0200)]
Split translate dump_unwindsyms in separate collection/dump phases.
Untangle separate module data collection. First collect all information,
build-ids, symbols, unwind tables, etc. Put all collected info in the
unwindsym_dump_context. Then dump it all out using the context.
Dave Brolley [Tue, 11 Oct 2011 18:49:07 +0000 (14:49 -0400)]
PR 13128: Modify unprivileged mode infrastructure to support a multi-privilege design
vs the previous boolean privileged/unprivileged design.
- in stap
- Store the specified privilege level as an enum as opposed to a boolean
- Test privilege levels against the proper enumerator.
- in the generated code
- Generate STP_PRIVILEGE macro with the specified privilege level
- Generated code tests it against the proper privilege level.
Josh Stone [Tue, 11 Oct 2011 23:55:59 +0000 (16:55 -0700)]
tracepoints: Add support for CONDITION events
These are events that check a condition before raising callbacks, but
take no hit for the condition when not tracing. We don't have to do
anything special in stap except handle the new macro, as the condition
is handled at the trace site.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): Add a
definition for DECLARE_TRACE_CONDITION. Also settle the macro-
arg wrappers to PARAMS().
Josh Stone [Tue, 11 Oct 2011 22:04:13 +0000 (15:04 -0700)]
tracepoints: Support reading enum parameters
* tapsets.cxx (resolve_tracepoint_arg_type): Treat enum like any other
numeric base type, simply casted to int64_t.
(tracepoint_derived_probe::build_args): Log parameter availability.
Josh Stone [Tue, 11 Oct 2011 21:15:27 +0000 (14:15 -0700)]
tracepoints: Skip identical headers from the build and source trees
In preparing the tracequery source, we normalize paths containing
"include/" to just the suffix part, so the header will be found by the
normal -I path. However, the debuginfo source tree usually contains
at least the same headers found in the build tree. We were thus
building effectively the same tracequery twice for headers that are
present in both the build and source trees. Stop that!
* tapsets.cxx (tracepoint_builder::init_dw): Build the header list with
knowledge of the "include/" normalization to avoid extra work.
Josh Stone [Tue, 11 Oct 2011 20:42:04 +0000 (13:42 -0700)]
tracepoints: Avoid building tracequery.ko targets
Since we're now only using the tracequery.o intermediates, we can forgo
building the tracequery.ko modules altogether. This saves about a third
of the processing time on my machine.
* buildrun.cxx (make_any_make_cmd): New, the common make_cmd base.
(make_make_cmd): The normal case targets "modules".
(make_make_objs_cmd): New, build only stage-1 objects.
(make_tracequeries): Call make_make_objs_cmd.
Josh Stone [Tue, 11 Oct 2011 20:21:34 +0000 (13:21 -0700)]
tracepoints: Work with the tracequery's .o rather than .ko
The intermediate object files are slightly smaller, since they don't
have any of the module boilerplate compiled in yet. It still has what
we need though, just the basic stapprobe_* function definitions.
* buildrun.cxx (make_tracequeries): Return .o filenames rather than .ko.
* hash.cxx (find_tracequery_hash): Use .o for cached filenames.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): Update
comments and variable names to refer to objects instead of kos.
Josh Stone [Tue, 11 Oct 2011 20:13:40 +0000 (13:13 -0700)]
tracepoints: Cache an empty file for failures
When a tracepoint header doesn't compile for any reason, we'd like not
to keep trying on subsequent runs. This patch restores previous
behavior that would save an empty file in the cache, signaling that
we've tried this header before and can't get anything from it.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): When reading
from the cache, note and skip empty files. When saving to the cache,
copy /dev/null for those that failed to compile.