Josh Stone [Fri, 28 Oct 2011 22:11:17 +0000 (15:11 -0700)]
Refactor sdt & plt pattern registration
This factors the sdt and plt suffixes away from the various process and
process.library prefixes. As a result, we also gain process.plt for
probing -c commands.
* tapsets.cxx (dwarf_derived_probe::register_sdt_variants): New.
(dwarf_derived_probe::register_plt_variants): New.
(dwarf_derived_probe::register_patterns): Register sdt and plt for all
process/library combinations.
* testsuite/semok/plt2.stp: Test that process.plt succeeds with -c
* testsuite/semko/plt2.stp: Test that process.plt fails without -c
Josh Stone [Fri, 28 Oct 2011 21:42:23 +0000 (14:42 -0700)]
plt: Throw an appropriate error for missing $var
* tapsets.cxx (plt_expanding_visitor::visit_target_symbol): Throw a
semantic_error for missing $var, so it may be caught and chained for
later. This ensures that the visitor always has a provide() too.
* testsuite/semko/plt1.stp: New, check that missing $var is an error.
* testsuite/semok/plt1.stp: New, check that $$name works, as does
@defined on a missing $var.
David Smith [Thu, 27 Oct 2011 15:24:45 +0000 (10:24 -0500)]
More PR13083 fixes to avoid sleeping in the task_finder.
* runtime/task_finder.c (__stp_get_mm_path): Call down_read_trylock()
instead of down_read() to avoid the possibility of sleeping.
(stap_start_task_finder): Call task_lock() to make sure a task's mm
doesn't go away while we're looking at it.
Based on peer feedback:
- Compare privilege credentials as sets, not levels.
- Send privilege credentials as a ctl message at startup.
- Send startup ctl messages (tzinfo, relocations, privilege_credentials)
on the same control channel. Ensure channel is closed properly on error.
Mark Wielaard [Fri, 21 Oct 2011 14:49:02 +0000 (16:49 +0200)]
Extract addr_of for unprivileged_[my]probes.exp once and with optional dot.
Some architectures prefix function symbols with a '.', so accept an
optional dot prefix. Also extract each symbol only once instead of
for each pass over the list.
Mark Wielaard [Fri, 21 Oct 2011 11:34:21 +0000 (13:34 +0200)]
Extend vma_map entry vm_end range when additional segments are mapped.
Once registered, we may want to extend an earlier registered region.
A segment might be mapped with different flags for different offsets.
If so we want to record the extended range so we can address more
precisely to module names and symbols.
David Smith [Wed, 19 Oct 2011 15:00:45 +0000 (10:00 -0500)]
Fixed PR13158 by adding a staprun signal handler so modules don't remain.
* runtime/staprun/staprun.c (term_signal_handler): New function.
(setup_term_signals): Ditto.
(insert_stap_module): Remembers if insert_module() worked.
(main): Call setup_term_signals().
* testsuite/systemtap.base/rename_module.exp: Made test case more robust
by waiting on output from all stap scripts.
* testsuite/systemtap.base/pr13158.exp: New testcase.
* transport/relay_v2.c, relayfs.c (_stp_transport_data_fs_init):
Increment _stp_allocated_[net_]memory with relay buffers.
* transport/ring_buffer.c: Ditto. Plus fix scaling of the incoming -sNN
value, so NN represents megabytes consistently.
Frank Ch. Eigler [Tue, 18 Oct 2011 20:58:15 +0000 (16:58 -0400)]
PR13317: improve parse error recovery heuristics
This change changes the 10 parse errors reported by brendang for iostat-scsi.stp
to just six actually salient ones.
* parse.cxx (parser::parse): Reorder error recovery to skip
everything until next top-level keyword. Wrap try {} in
while {} rather than vice versa.
(parser::parse_stmt_block): Drop recovery loop.
Dave Brolley [Tue, 18 Oct 2011 17:54:20 +0000 (13:54 -0400)]
PR 13128: Pass user privilege credentials from staprun to the module.
- User's privilege credentials are determined by staprun and passed to the module
- Module contains generated code which checks the user's credentials against those
required to load/run the module (specified at compile time).
- module aborts if credentials are insufficient.
Mark Wielaard [Tue, 18 Oct 2011 12:28:59 +0000 (14:28 +0200)]
PR13284 Failure to resolve kernel global e.g. jiffies.
* dwflpp.cxx (vardie_from_symtable): Also accept STT_NOTYPE. Some global
don't have a type associated with them.
* testsuite/buildok/pr13284.stp: New testcase.
Josh Stone [Mon, 17 Oct 2011 22:18:06 +0000 (15:18 -0700)]
PR13283: Use a temporary string for comparisons
The reported issue was trying to directly compare the results of two
different kernel_string() calls, but they were always computed equal
even though they were known to be different. Inspection of pass-3
showed that both calls led to an expr-stmt ending in ...->__retvalue.
This is just a pointer though, and it happens here that both calls will
point to the same string buffer, so strncmp rightly calls them equal.
To fix this, we must introduce temporary storage for the intermediate
strings. Now we won't lose either value, and they can be properly
compared for equality.
* translate.cxx (c_tmpcounter::visit_comparison): Declare temporary
storage for the left operand if it's not a literal.
(c_unparser::visit_comparison): Use the left temporary.
* testsuite/systemtap.string/strcmp.*: Test string comparisons.
Josh Stone [Thu, 13 Oct 2011 22:55:24 +0000 (15:55 -0700)]
Tweak cache logic to group removals by hash value
We used to glob the cache for *.ko, *.so, *.h, and *.log, implicitly
assuming that this was all that was cached. For the .ko case we also
grouped .c and .sgn to be removed all together. But when any new
extension came along, the cache cleanup ignored it.
Now we glob instead for any file in the cache that looks like is has a
hashed name, and we group removals for all files that appear to have the
same hash value, regardless of prefix or suffix. This will hopefully
make us more robust against future file-caching changes.
* cache.cxx (cache_ent_info): Becomes a vector of paths, with no
knowledge needed for modules. Use better types for size and mtime.
(clean_cache): Use a new glob that looks for hash-like files, with 32
hex digits in a row. Use a regex to find common HASH_LEN substrings
in all those filenames. Build the cache_ent_info groups by HASH_LEN,
and proceed to delete old entries until below quota.
Mark Wielaard [Thu, 13 Oct 2011 21:19:21 +0000 (23:19 +0200)]
Handle PPC64 function symbols by resolving function descriptors.
* translate.cxx (dump_symbol_table): If the module is EM_PPC64, but not
ET_REL, then register two address for a STT_FUNC. The function descriptor
address and the address of the function that the descriptor points to.
For ET_REL still rely on the .function_name convention.
David Smith [Thu, 13 Oct 2011 21:24:43 +0000 (16:24 -0500)]
(More PR13289 fixes) Added better %m/%M error handling.
* runtime/print.c (_stp_unreserve_bytes): New function.
* runtime/print.h: Added _stp_unreserve_bytes() declaration.
* runtime/vsprintf.c (_stp_vsprint_memory): Only change the input pointer
to "<NULL>" when we're formatting a string (%s), not when we're
formatting raw memory (%m/%M). Also, return NULL on errors.
(_stp_vsprint_memory_size): Only change the input pointer to "<NULL>"
when we're formatting a string (%s), not when we're formatting raw
memory (%m/%M).
(_stp_vsnprintf): If _stp_vsprint_memory() returns NULL, unreserve bytes
in the print buffer, when necessary.
* translate.cxx (c_unparser::emit_compiled_printfs): If
_stp_vsprint_memory() returns NULL, unreserve bytes in the print buffer,
when necessary. Also improve error message if _stp_vsprint_memory()
fails by setting 'last_stmt'.
* testsuite/systemtap.stress/conversions.stp: Added tests for '%M' and for
using sprintf() with '%m' and '%M'.
* testsuite/systemtap.stress/conversions.exp: Updated error count.
Josh Stone [Thu, 13 Oct 2011 20:22:29 +0000 (13:22 -0700)]
PR13155: Fix tracepoint argument access
The restructured tracepoint build broke the way tracepoint parameters
are passed to the script. This patch cleans that up, and simplifies a
few other parts of the code generation.
* tapsets.cxx (tracepoint_derived_probe_group::emit_module_decls):
Define intptr_t for the aux source. Collect .used entries from
p->args[] into used_args[] to ease call-list generation. Simplify
registration by replacing the wrappers with plain declarations.
* buildrun.cxx (compile_pass): We do have everything needed for strict
prototypes, so don't squash this warning.
Frank Ch. Eigler [Thu, 13 Oct 2011 16:33:48 +0000 (12:33 -0400)]
PR13155: separate compilation for tracepoint-related functions
The linux kernel's tracepoint headers are sometimes mutually
conflicting, defining structs etc. differently. Since we need to use
each header in its natural habitat, we need to separately-compile
those tracepoint-related functions that absolutely need access to
those declarations. This is quite new to us as so far, the translator
has usually emitted a single translated_source output, plus the
stap-symbols.h file.
* buildrun.cxx (compile_pass): Compile auxiliary sources into main module.
* tapsets.cxx (tracepoint_derived_probe_group::emit_module_decls): Effectively
rewrite to create per-tracepoint-header auxiliary source files.
* session.cxx (op_create_auxiliary): New function.
* translate.cxx (translator_output): Add filename field.
* session.h, translate.h: Corresponding changes.
* hash.cxx (find_script_hash): Rename primary source file to stap_XXX_src.c.
* main.cxx (passes_0_4): Ditto.
David Smith [Thu, 13 Oct 2011 17:37:29 +0000 (12:37 -0500)]
Fixed PR13083 by no longer calling mmput() when possible.
* runtime/task_finder.c (__stp_utrace_attach): Don't call
get_task_mm()/mmput(). Since we're in the context of that task, the mm
should stick around (and mmput() can sleep).
(__stp_call_mmap_callbacks_with_addr): Ditto.
(__stp_utrace_attach_match_tsk): Ditto.
(__stp_call_mmap_callbacks_for_task): Ditto.
Chris Meek [Thu, 13 Oct 2011 17:04:54 +0000 (13:04 -0400)]
PR13261: Added map->wrap mode to script parser
To activate map->wrap mode, use the '%' symbol after the variable name
in a global declaration, for example:
global foo%[100]
This will let it automatically replace elements in the map if you
try to place something out of bounds, rather than give an error.
It also works for statistics-valued arrays.
parse.cxx
Checked for the '%' symbol in parser::parse_global(), and if
present, sets the vardecl* d->wrap value to true.
staptree.cxx
Set the default value of the new vardecl->wrap boolean value to false.
Added the '%' to the hash if it is enabled.
staptree.h
Added the bool wrap element to struct vardecl
translate.cxx
Added the bool wrap element to struct mapvar and initialixed it to the
wrap element of the associated vardecl variable.
Checked whether the mapvar->wrap has been set (from vardecl->wrap set
in parser::parse_global()).
If so, added the apropriate code to set the map->wrap for whether its a
map or a pmap.
Mark Wielaard [Thu, 13 Oct 2011 11:53:08 +0000 (13:53 +0200)]
PR13241 too much dump-unwindsyms processing for modules even if !need_symbols
Introduce dump_section_list() that will pregenerate the section list when
needed. dump_symbol_tables() now does the minimum necessary to extract
the needed symbols, or will not even be called at all when !need_symbols
and the module isn't the "kernel".
David Smith [Wed, 12 Oct 2011 21:52:30 +0000 (16:52 -0500)]
Fixed PR12389 by making %m/%M use kread() to access memory.
* runtime/vsprintf.c (_stp_vsprint_memory): The '%m' and '%M' formatting
operators use kread() to read memory.
* translate.cxx: Removed all support from the c_unparser class for
'probe_or_function_needs_deref_fault_handler' processing since it is no
longer needed.
(c_unparser::visit_print_format): Removed deref_buffer() call. Added
check for c->last_error after print is done.
* runtime/runtime_context.h: New file.
* runtime/vsprintf.h: Ditto.
* runtime/print.c: Removed extra include of vsprintf.c. Allow use to
override STP_LOG_BUF_LEN. Make _stp_print_flush() definitions exactly
the same between the RELAY_{GUEST,HOST} and normal caes.
* runtime/print.h: Added a few more functions and includes the new
vsprintf.h file.
* runtime/runtime.h: Includes print.h instead of print.c. Moved print.c
and io.c to runtime_context.h.
* runtime/transport/transport.h: Declares '_stp_target'.
Mark Wielaard [Wed, 12 Oct 2011 10:53:30 +0000 (12:53 +0200)]
Split translate dump_unwindsyms in separate collection/dump phases.
Untangle separate module data collection. First collect all information,
build-ids, symbols, unwind tables, etc. Put all collected info in the
unwindsym_dump_context. Then dump it all out using the context.
Dave Brolley [Tue, 11 Oct 2011 18:49:07 +0000 (14:49 -0400)]
PR 13128: Modify unprivileged mode infrastructure to support a multi-privilege design
vs the previous boolean privileged/unprivileged design.
- in stap
- Store the specified privilege level as an enum as opposed to a boolean
- Test privilege levels against the proper enumerator.
- in the generated code
- Generate STP_PRIVILEGE macro with the specified privilege level
- Generated code tests it against the proper privilege level.
Josh Stone [Tue, 11 Oct 2011 23:55:59 +0000 (16:55 -0700)]
tracepoints: Add support for CONDITION events
These are events that check a condition before raising callbacks, but
take no hit for the condition when not tracing. We don't have to do
anything special in stap except handle the new macro, as the condition
is handled at the trace site.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): Add a
definition for DECLARE_TRACE_CONDITION. Also settle the macro-
arg wrappers to PARAMS().
Josh Stone [Tue, 11 Oct 2011 22:04:13 +0000 (15:04 -0700)]
tracepoints: Support reading enum parameters
* tapsets.cxx (resolve_tracepoint_arg_type): Treat enum like any other
numeric base type, simply casted to int64_t.
(tracepoint_derived_probe::build_args): Log parameter availability.
Josh Stone [Tue, 11 Oct 2011 21:15:27 +0000 (14:15 -0700)]
tracepoints: Skip identical headers from the build and source trees
In preparing the tracequery source, we normalize paths containing
"include/" to just the suffix part, so the header will be found by the
normal -I path. However, the debuginfo source tree usually contains
at least the same headers found in the build tree. We were thus
building effectively the same tracequery twice for headers that are
present in both the build and source trees. Stop that!
* tapsets.cxx (tracepoint_builder::init_dw): Build the header list with
knowledge of the "include/" normalization to avoid extra work.
Josh Stone [Tue, 11 Oct 2011 20:42:04 +0000 (13:42 -0700)]
tracepoints: Avoid building tracequery.ko targets
Since we're now only using the tracequery.o intermediates, we can forgo
building the tracequery.ko modules altogether. This saves about a third
of the processing time on my machine.
* buildrun.cxx (make_any_make_cmd): New, the common make_cmd base.
(make_make_cmd): The normal case targets "modules".
(make_make_objs_cmd): New, build only stage-1 objects.
(make_tracequeries): Call make_make_objs_cmd.
Josh Stone [Tue, 11 Oct 2011 20:21:34 +0000 (13:21 -0700)]
tracepoints: Work with the tracequery's .o rather than .ko
The intermediate object files are slightly smaller, since they don't
have any of the module boilerplate compiled in yet. It still has what
we need though, just the basic stapprobe_* function definitions.
* buildrun.cxx (make_tracequeries): Return .o filenames rather than .ko.
* hash.cxx (find_tracequery_hash): Use .o for cached filenames.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): Update
comments and variable names to refer to objects instead of kos.
Josh Stone [Tue, 11 Oct 2011 20:13:40 +0000 (13:13 -0700)]
tracepoints: Cache an empty file for failures
When a tracepoint header doesn't compile for any reason, we'd like not
to keep trying on subsequent runs. This patch restores previous
behavior that would save an empty file in the cache, signaling that
we've tried this header before and can't get anything from it.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): When reading
from the cache, note and skip empty files. When saving to the cache,
copy /dev/null for those that failed to compile.
* tapsets.cxx (tracepoint_extra_decls): Prepare for PR13155.
(get_tracepoint_modules): Rewrite former get_tracepoint_module().
(tracepoint_builder::init_dw): Don't even try to build all tracequery
modules at once.
* buildrun.cxx (make_tracequeries): Replace make_tracequery().
Run given compilation jobs in parallel.
* buildrun.h: Corresponding changes.
* hash.cxx (find_tracequery_hash): Clarify signature for single header.
* hash.h: Corresponding changes.
Josh Stone [Fri, 7 Oct 2011 17:13:38 +0000 (10:13 -0700)]
PR3217: Fix systemtap.base/vars.exp
This test has a grep on pass-3 code, which started matching too much
with the new compiled-printf output. This tightens that grep down a
bit, and adds a comment explaining what the test is trying to do.
Josh Stone [Fri, 7 Oct 2011 16:48:43 +0000 (09:48 -0700)]
PR13272: Fix quoted strings in the printf shortcut path
For printf strings with no arguments, we usually skip all the printf
path altogether and just use _stp_print. However, the compiled-printf
changes affected the level of string escaping, which broke for a case
like printf("\"quote\\this\"\n"). We must make sure that this shortcut
path uses an output-quoted string literal.
* translate.cxx (c_unparser::visit_print_format): Generate an output-
quoted format_string too, for use by the _stp_print shortcut.
(preprocess_print_format): The "%s\n" case shouldn't alter components.
* testsuite/buildok/printf.stp: Add "%s" and "%s\n" shortcut tests too.
Mark Wielaard [Fri, 7 Oct 2011 09:51:06 +0000 (11:51 +0200)]
Always remove prefix '.' when _stp_snprint_addr() human readable symname.
Some arches prefix some symbol names with '.', to make symbol names as
presented in human readable form consistent across arches always remove
them when printing them with _stp_snprint_addr. Remove special casing
in context-symbols.stp probefunc().
Josh Stone [Thu, 6 Oct 2011 20:00:09 +0000 (13:00 -0700)]
PR3217: Add STP_LEGACY_PRINT option to use the old form
* translate.cxx (c_unparser::visit_print_format): Alternate using
STP_LEGACY_PRINT between the new compiled printfs and the old
_stp_[s]printf, so we can make sure both still work.
(c_unparser::emit_compiled_printf_locals): Conditionalize.
(c_unparser::emit_compiled_printfs): Conditionalize.
* testsuite/lib/stap_run2.exp: Show extra args used in stap_run2.
* testsuite/systemtap.printf/*.exp: Run -DSTP_LEGACY_PRINT too in all
the tests using stap_run2. Others will require greater restructuring
to achieve this...
Josh Stone [Thu, 6 Oct 2011 19:33:59 +0000 (12:33 -0700)]
PR3217: Make compiled-printf functions noinline
When running systemtap.printf/int1.stp, I got:
...: In function ‘probe_1997’:
...:3885:1: error: the frame size of 772 bytes is larger than 256 bytes [-Werror=frame-larger-than=]
It's not yet clear to me why gcc claims so much stack space here, as
even in the compiled printf functions, most of the local variables have
very limited scope. For now, marking them noinline solves the issue,
but this deserves closer inspection...
* translate.cxx (c_unparser::emit_compiled_printfs): Mark each generated
printf function as noinline.
Josh Stone [Tue, 27 Sep 2011 22:30:13 +0000 (15:30 -0700)]
PR3217: Merge similar compiled-printfs
It's pretty common for tapset wildcards to expand to lots of probes with
slightly different handlers (e.g. syscall.*), and we'd like for the
newly expanded printf code to not be duplicated so much.
This patch keeps a hash in the translator of visited format strings, so
we can output a compiled version of that only once. This requires some
new argument space in the context, but we can optimize by realizing that
(1) printfs never nest when arguments are precomputed, (2) string
arguments are immutable and need not be copied, and (3) return strings
for sprintf can write directly to the caller's tmpvar.
* translate.cxx (c_unparser::compiled_printfs): New map for storing
pair(print_to_stream, format_string) -> compiled printf name.
(c_unparser::emit_compiled_printfs): New, emit the expanded function
body for every compiled printf.
(c_unparser::emit_compiled_printf_locals): New, emit a union-struct of
context locals for all compiled printfs.
(c_unparser::emit_common_header): Call emit_compiled_printf*.
(c_unparser::declare_compiled_printf): New, called to declare that we
need this printf -- add it to the map if it's not already there.
(c_tmpcounter::visit_print_format): Declare printfs as we see them.
(c_unparser::get_compiled_printf): New, get the name for a previously
declared printf function.
(c_unparser::visit_print_format): Get the compiled printf function.
Josh Stone [Tue, 7 Jun 2011 23:50:20 +0000 (16:50 -0700)]
PR3217: Compiled printf
This translates script-level printf statements into unrolled printing
statements. The compiled code is faster, but also larger.
* runtime/vsprintf.c (_stp_vsprint_memory): Factored out for %s/%m/%M.
(_stp_vsprint_memory_size): Factored out for size of %s/%m/%M.
(_stp_vsprint_memory_size): Factored out for %b.
(_stp_vsprint_binary_size): Factored out for size of %b.
* staptree.h (print_format::format_flag): Match the same definitions
used by print_flag in runtime/vsprintf.c.
(print_format::conversion_type): Collapse numeric types into one.
(print_format::format_component::base): New field to classify numeric.
(print_format::set_flag, print_format::test_flag): New, for convenience.
* elaborate.cxx (typeresolution_info::visit_print_format): Handle the
changes in conversion_type.
* staptree.cxx (print_format::components_to_string): Parse out the new
and impoved format_component.
(print_format::string_to_components): Create the new format_component.
* translate.cxx (c_unparser::visit_print_format): Generate unrolled code
for printf statements.
Mark Wielaard [Wed, 5 Oct 2011 13:04:17 +0000 (15:04 +0200)]
Support same value rules in unwinder. Don't treat undefined as same value.
The unwinder treated undefined and same value rules the same. This patch
introduces different states for them. All "real" registers are assumed to
be "same value". Their value will be carried over to the next frame unless
overruled. All other "fake" dwarf registers are treated as undefined.
This makes a difference for architectures where for example the dwarf
return register is a real register, where same value means something
different from undefined (e.g. ppc).
Mark Wielaard [Tue, 4 Oct 2011 13:19:52 +0000 (15:19 +0200)]
Introduce DWARF_MAP_REG for unwind.
The reg_info array assumes dwarf register numbers start at zero and
are consecutive. If that isn't the case for some architecture (e.g. ppc)
then redefine DWARF_MAP_REG to map the given dwarf register number to the
actual reg_info index.
Stan Cox [Mon, 3 Oct 2011 15:14:44 +0000 (11:14 -0400)]
Add process.library.plt
tapsets.cxx (symbol_table::get_first_func): New.
(dwarf_query::query_module_symtab): Use it to as a placeholder, which
is replaced by plt entry address.
(symbol_table::*) Add \n to cerr messages.
Mark Wielaard [Fri, 30 Sep 2011 13:57:43 +0000 (15:57 +0200)]
Check module was actually created in tracepoint_builder.
Commit e86f32 added -i to the make invocation for tracequery jobs. This
means make would succeed even if there were errors. So it was possible
that tracequery_ko didn't actually exist. Leading to spurious warnings
about copying non-existing files into the cache when used.
tapset: remove result caching from __find_bdevname
This function is indirectly used from many vfs tapset probe points.
Since it sets/gets an internal script global variable, the current
optimizer does not elide this function, thus wasting time, space, and
risks @cast/kread runtime errors. Let's get rid of the cache, until
the optimizer can do this automagically.