Dave Brolley [Tue, 11 Oct 2011 18:49:07 +0000 (14:49 -0400)]
PR 13128: Modify unprivileged mode infrastructure to support a multi-privilege design
vs the previous boolean privileged/unprivileged design.
- in stap
- Store the specified privilege level as an enum as opposed to a boolean
- Test privilege levels against the proper enumerator.
- in the generated code
- Generate STP_PRIVILEGE macro with the specified privilege level
- Generated code tests it against the proper privilege level.
Josh Stone [Tue, 11 Oct 2011 23:55:59 +0000 (16:55 -0700)]
tracepoints: Add support for CONDITION events
These are events that check a condition before raising callbacks, but
take no hit for the condition when not tracing. We don't have to do
anything special in stap except handle the new macro, as the condition
is handled at the trace site.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): Add a
definition for DECLARE_TRACE_CONDITION. Also settle the macro-
arg wrappers to PARAMS().
Josh Stone [Tue, 11 Oct 2011 22:04:13 +0000 (15:04 -0700)]
tracepoints: Support reading enum parameters
* tapsets.cxx (resolve_tracepoint_arg_type): Treat enum like any other
numeric base type, simply casted to int64_t.
(tracepoint_derived_probe::build_args): Log parameter availability.
Josh Stone [Tue, 11 Oct 2011 21:15:27 +0000 (14:15 -0700)]
tracepoints: Skip identical headers from the build and source trees
In preparing the tracequery source, we normalize paths containing
"include/" to just the suffix part, so the header will be found by the
normal -I path. However, the debuginfo source tree usually contains
at least the same headers found in the build tree. We were thus
building effectively the same tracequery twice for headers that are
present in both the build and source trees. Stop that!
* tapsets.cxx (tracepoint_builder::init_dw): Build the header list with
knowledge of the "include/" normalization to avoid extra work.
Josh Stone [Tue, 11 Oct 2011 20:42:04 +0000 (13:42 -0700)]
tracepoints: Avoid building tracequery.ko targets
Since we're now only using the tracequery.o intermediates, we can forgo
building the tracequery.ko modules altogether. This saves about a third
of the processing time on my machine.
* buildrun.cxx (make_any_make_cmd): New, the common make_cmd base.
(make_make_cmd): The normal case targets "modules".
(make_make_objs_cmd): New, build only stage-1 objects.
(make_tracequeries): Call make_make_objs_cmd.
Josh Stone [Tue, 11 Oct 2011 20:21:34 +0000 (13:21 -0700)]
tracepoints: Work with the tracequery's .o rather than .ko
The intermediate object files are slightly smaller, since they don't
have any of the module boilerplate compiled in yet. It still has what
we need though, just the basic stapprobe_* function definitions.
* buildrun.cxx (make_tracequeries): Return .o filenames rather than .ko.
* hash.cxx (find_tracequery_hash): Use .o for cached filenames.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): Update
comments and variable names to refer to objects instead of kos.
Josh Stone [Tue, 11 Oct 2011 20:13:40 +0000 (13:13 -0700)]
tracepoints: Cache an empty file for failures
When a tracepoint header doesn't compile for any reason, we'd like not
to keep trying on subsequent runs. This patch restores previous
behavior that would save an empty file in the cache, signaling that
we've tried this header before and can't get anything from it.
* tapsets.cxx (tracepoint_builder::get_tracequery_modules): When reading
from the cache, note and skip empty files. When saving to the cache,
copy /dev/null for those that failed to compile.
* tapsets.cxx (tracepoint_extra_decls): Prepare for PR13155.
(get_tracepoint_modules): Rewrite former get_tracepoint_module().
(tracepoint_builder::init_dw): Don't even try to build all tracequery
modules at once.
* buildrun.cxx (make_tracequeries): Replace make_tracequery().
Run given compilation jobs in parallel.
* buildrun.h: Corresponding changes.
* hash.cxx (find_tracequery_hash): Clarify signature for single header.
* hash.h: Corresponding changes.
Josh Stone [Fri, 7 Oct 2011 17:13:38 +0000 (10:13 -0700)]
PR3217: Fix systemtap.base/vars.exp
This test has a grep on pass-3 code, which started matching too much
with the new compiled-printf output. This tightens that grep down a
bit, and adds a comment explaining what the test is trying to do.
Josh Stone [Fri, 7 Oct 2011 16:48:43 +0000 (09:48 -0700)]
PR13272: Fix quoted strings in the printf shortcut path
For printf strings with no arguments, we usually skip all the printf
path altogether and just use _stp_print. However, the compiled-printf
changes affected the level of string escaping, which broke for a case
like printf("\"quote\\this\"\n"). We must make sure that this shortcut
path uses an output-quoted string literal.
* translate.cxx (c_unparser::visit_print_format): Generate an output-
quoted format_string too, for use by the _stp_print shortcut.
(preprocess_print_format): The "%s\n" case shouldn't alter components.
* testsuite/buildok/printf.stp: Add "%s" and "%s\n" shortcut tests too.
Mark Wielaard [Fri, 7 Oct 2011 09:51:06 +0000 (11:51 +0200)]
Always remove prefix '.' when _stp_snprint_addr() human readable symname.
Some arches prefix some symbol names with '.', to make symbol names as
presented in human readable form consistent across arches always remove
them when printing them with _stp_snprint_addr. Remove special casing
in context-symbols.stp probefunc().
Josh Stone [Thu, 6 Oct 2011 20:00:09 +0000 (13:00 -0700)]
PR3217: Add STP_LEGACY_PRINT option to use the old form
* translate.cxx (c_unparser::visit_print_format): Alternate using
STP_LEGACY_PRINT between the new compiled printfs and the old
_stp_[s]printf, so we can make sure both still work.
(c_unparser::emit_compiled_printf_locals): Conditionalize.
(c_unparser::emit_compiled_printfs): Conditionalize.
* testsuite/lib/stap_run2.exp: Show extra args used in stap_run2.
* testsuite/systemtap.printf/*.exp: Run -DSTP_LEGACY_PRINT too in all
the tests using stap_run2. Others will require greater restructuring
to achieve this...
Josh Stone [Thu, 6 Oct 2011 19:33:59 +0000 (12:33 -0700)]
PR3217: Make compiled-printf functions noinline
When running systemtap.printf/int1.stp, I got:
...: In function ‘probe_1997’:
...:3885:1: error: the frame size of 772 bytes is larger than 256 bytes [-Werror=frame-larger-than=]
It's not yet clear to me why gcc claims so much stack space here, as
even in the compiled printf functions, most of the local variables have
very limited scope. For now, marking them noinline solves the issue,
but this deserves closer inspection...
* translate.cxx (c_unparser::emit_compiled_printfs): Mark each generated
printf function as noinline.
Josh Stone [Tue, 27 Sep 2011 22:30:13 +0000 (15:30 -0700)]
PR3217: Merge similar compiled-printfs
It's pretty common for tapset wildcards to expand to lots of probes with
slightly different handlers (e.g. syscall.*), and we'd like for the
newly expanded printf code to not be duplicated so much.
This patch keeps a hash in the translator of visited format strings, so
we can output a compiled version of that only once. This requires some
new argument space in the context, but we can optimize by realizing that
(1) printfs never nest when arguments are precomputed, (2) string
arguments are immutable and need not be copied, and (3) return strings
for sprintf can write directly to the caller's tmpvar.
* translate.cxx (c_unparser::compiled_printfs): New map for storing
pair(print_to_stream, format_string) -> compiled printf name.
(c_unparser::emit_compiled_printfs): New, emit the expanded function
body for every compiled printf.
(c_unparser::emit_compiled_printf_locals): New, emit a union-struct of
context locals for all compiled printfs.
(c_unparser::emit_common_header): Call emit_compiled_printf*.
(c_unparser::declare_compiled_printf): New, called to declare that we
need this printf -- add it to the map if it's not already there.
(c_tmpcounter::visit_print_format): Declare printfs as we see them.
(c_unparser::get_compiled_printf): New, get the name for a previously
declared printf function.
(c_unparser::visit_print_format): Get the compiled printf function.
Josh Stone [Tue, 7 Jun 2011 23:50:20 +0000 (16:50 -0700)]
PR3217: Compiled printf
This translates script-level printf statements into unrolled printing
statements. The compiled code is faster, but also larger.
* runtime/vsprintf.c (_stp_vsprint_memory): Factored out for %s/%m/%M.
(_stp_vsprint_memory_size): Factored out for size of %s/%m/%M.
(_stp_vsprint_memory_size): Factored out for %b.
(_stp_vsprint_binary_size): Factored out for size of %b.
* staptree.h (print_format::format_flag): Match the same definitions
used by print_flag in runtime/vsprintf.c.
(print_format::conversion_type): Collapse numeric types into one.
(print_format::format_component::base): New field to classify numeric.
(print_format::set_flag, print_format::test_flag): New, for convenience.
* elaborate.cxx (typeresolution_info::visit_print_format): Handle the
changes in conversion_type.
* staptree.cxx (print_format::components_to_string): Parse out the new
and impoved format_component.
(print_format::string_to_components): Create the new format_component.
* translate.cxx (c_unparser::visit_print_format): Generate unrolled code
for printf statements.
Mark Wielaard [Wed, 5 Oct 2011 13:04:17 +0000 (15:04 +0200)]
Support same value rules in unwinder. Don't treat undefined as same value.
The unwinder treated undefined and same value rules the same. This patch
introduces different states for them. All "real" registers are assumed to
be "same value". Their value will be carried over to the next frame unless
overruled. All other "fake" dwarf registers are treated as undefined.
This makes a difference for architectures where for example the dwarf
return register is a real register, where same value means something
different from undefined (e.g. ppc).
Mark Wielaard [Tue, 4 Oct 2011 13:19:52 +0000 (15:19 +0200)]
Introduce DWARF_MAP_REG for unwind.
The reg_info array assumes dwarf register numbers start at zero and
are consecutive. If that isn't the case for some architecture (e.g. ppc)
then redefine DWARF_MAP_REG to map the given dwarf register number to the
actual reg_info index.
Stan Cox [Mon, 3 Oct 2011 15:14:44 +0000 (11:14 -0400)]
Add process.library.plt
tapsets.cxx (symbol_table::get_first_func): New.
(dwarf_query::query_module_symtab): Use it to as a placeholder, which
is replaced by plt entry address.
(symbol_table::*) Add \n to cerr messages.
Mark Wielaard [Fri, 30 Sep 2011 13:57:43 +0000 (15:57 +0200)]
Check module was actually created in tracepoint_builder.
Commit e86f32 added -i to the make invocation for tracequery jobs. This
means make would succeed even if there were errors. So it was possible
that tracequery_ko didn't actually exist. Leading to spurious warnings
about copying non-existing files into the cache when used.
tapset: remove result caching from __find_bdevname
This function is indirectly used from many vfs tapset probe points.
Since it sets/gets an internal script global variable, the current
optimizer does not elide this function, thus wasting time, space, and
risks @cast/kread runtime errors. Let's get rid of the cache, until
the optimizer can do this automagically.
Mark Wielaard [Wed, 28 Sep 2011 11:18:18 +0000 (13:18 +0200)]
Introduce gfp_mask variants of _stp allocation functions.
* runtime/alloc.c: Add _stp_kmalloc_gfp(), _stp_kzalloc_gfp() and
_stp_kmalloc_node_gfp() which take a gfp_mask in case you really know
what you are doing.
* translate.cxx (emit_module_init): Use with GFP_KERNEL for context alloc.
* runtime/map.c: Use for init and new functions called at module_init time.
* runtime/stat.c: Likewise.
Mark Wielaard [Tue, 27 Sep 2011 14:21:15 +0000 (16:21 +0200)]
_stp_vma_match_vdso: Don't subtract build_id_len from notes_addr.
Older (pre-0.138) elfutils would erroneously return the address of the
build id note address with the length added. Newer ones never do this.
And we only support elfutils 0.142+ now anyway. Fixes vma_vdso.exp.
Mark Wielaard [Wed, 21 Sep 2011 10:38:24 +0000 (12:38 +0200)]
PR13210 vma/vdso tracking is broken testcase
Add a generic testcase to see if we can track the vma from which a syscall
was made. This might or might not come in through the vdso on the particular
(compat) arch, but should always at least be able to get the vma name.
Currently it fails some of the tests in different ways for different arches
and/or kernel versions.
Stan Cox [Tue, 20 Sep 2011 18:14:46 +0000 (14:14 -0400)]
Add process.plt test.
testsuite/systemtap.base/plt.exp: New test.
testsuite/systemtap.base/plt.c: New test case.
tapsets.cxx (base_query::base_query): .plt requires the elf symbol table.
Mark Wielaard [Tue, 20 Sep 2011 20:22:14 +0000 (22:22 +0200)]
Put struct unwind_frame_info in unwind.h and remove unused task field.
Remove the identical definitions of struct unwind_frame_info from
unwind/i386.h and unwind/x86_64.h and put it in unwind/unwind.h.
Also put struct unwind_context there to keep all struct definitions
together. Remove unused task field from struct unwind_frame_info.
Dave Brolley [Tue, 20 Sep 2011 16:56:42 +0000 (12:56 -0400)]
PR 13107: simplify compile-server testsuite
- Removed the use of a wrapper script to run stap using the server.
- Client/server tests now use the --use-server option as needed.
- testsuite/systemtap.server/server.exp is now reduced.
- no longer runs all of buildok tests
- now runs tests on server-specific functionality
- general client/server interoperability covered by existing tests.
With the previous code, it was possible for the delayedkill ./loop job
to be killed before the stap script watching for its end got really
ready. That could mean that the stap script starts, and waits, and
waits, and waits more, and waits, and waits, and waits, and waits, and
waits, and waits, and waits, and waits, and waits, and waits, and
waits, and waits, and waits yet more, and waits, and waits, and waits,
and waits, and waits, and waits, and waits, and waits, and waits, and
waits, and waits a bit longer, and waits, and waits, and waits, and
waits, and waits, and waits, and waits, and waits more , and waits,
and waits, and waits, and waits, and waits, and waits, and waits, and
waits, and keeps waiting, and waits, and waits, and waits, and waits,
and waits, and waits, and waits, and goes on waiting, and waits, and
waits a lot, and waits, and waits, and waits, and waits, and waits,
and waits, and waits, and waits for warts, and waits, and waits, and
waits, and waits, and waits, and waits, and waits, and waits, now
getting slightly impatient, and waits, and waits, and waits, and
waits, and waits, and waits, and waits, and waits, and waits, and
waits, and waits, and waits, and waits, and waits a lot more.
Since there is no robust synchronization in the test case, let's
increase the delayedkill interval to make the above wait less likely,
plus add a godot timeout probe that will abort the script before it
gets grey hairs, loses its teeth, and gets bombarded with funeral
pre-planning spam.
Mark Wielaard [Tue, 20 Sep 2011 09:48:28 +0000 (11:48 +0200)]
Introduce _stp_get_uregs and _STP_PROBE_STATE_FULL_UREGS flag.
No actual functionality change yet. _STP_PROBE_STATE_FULL_UREGS
is set for context probe_flags if full user register set is available
after calling _stp_get_uregs().
PR13200: skip prologue searching for *.function(ADDR).return probes
Prologue searching has to be suppressed for all kretprobes/uretprobes
address calculations, or else the *retprobes facility can overwrite
valid stack frame data instead of return addresses.
* tapsets.cxx (query_addr, query_func_info): Perform prologue analysis
but verbosely ignore results, for .return probes.
Mark Wielaard [Mon, 19 Sep 2011 13:42:19 +0000 (15:42 +0200)]
Remove arch specific _stp_probing_32bit_app() and _stp_get_regparm().
Both functions were doing more than necessary and didn't actually need
access to the current regs. _STP_PROBE_STATE_USER_MODE tells us whether
the current probe triggered in user mode. _stp_is_compat_task() tells
us whether the current task is a 32-on-64 process.
* runtime/compatdefs.h (_stp_is_compat_task): Defined also when CONFIG_COMPAT
is undefined (to always return false).
* runtime/loc2c-runtime.h: u_fetch_register() and u_store_register() are
only called from user space dwarf context. Only needs _stp_is_compat_task()
test.
* runtime/regs.c: Remove arch specific _stp_probing_32bit_app() and
_stp_get_regparm() functions.
* tapset/i386/registers.stp (_stp_arg): Check regparm directly.
* tapset/powerpc/registers.stp (probing_32bit_app): Use _stp_is_compat_task()
test.
* tapset/s390/registers.stp (probing_32bit_app): Likewise.
* tapset/x86_64/registers.stp (_stp_arg): Check regparm directly.
(probing_32bit_app): Use _stp_is_compat_task() test.
(regparm): Likewise.
Josh Stone [Fri, 16 Sep 2011 23:45:15 +0000 (16:45 -0700)]
PR12136: Test a @cast to type sdt::string
This test would previously fail to find std::string, as that type is not
at the top level of the CU in the debuginfo, but rather nested under the
DIE for the std namespace. It should now be found.
Note however that this test is currently failing for me due to an
apparently bad argument calculation. The @cast in the SDT probe looks
good, but from the function probe it's getting a bad pointer for $str.
It's OK on F15 i686, but not on F15 x86_64. I think this issue is
distinct from what I'm fixing, but it need further investigation...
Josh Stone [Fri, 16 Sep 2011 22:59:45 +0000 (15:59 -0700)]
Fill in @cast default module from SDT
If a @cast in SDT context does not have a specified source for type
info, then we should infer it as the SDT's module, as we do for normal
DWARF-based probes.
* tapsets.cxx (sdt_uprobe_var_expanding_visitor::visit_cast_op): If the
@cast module isn't already set, use the SDT module.
(sdt_kprobe_var_expanding_visitor::visit_cast_op): Ditto.
Josh Stone [Wed, 24 Aug 2011 23:21:38 +0000 (16:21 -0700)]
PR12136: Load nested types for C++ CUs
For C++ compile units only, check for nested type definitions within
namespaces, structures, and classes. This allows @cast to have access
to types like "std::string".
* dwflpp.cxx (dwflpp::global_alias_caching_callback): Take a flag
has_inner_types to indicate that we should recurse, and a prefix to
specify what we've already recursed into.
(dwflpp::iterate_over_globals): Check whether we're in C++, then do
the real work in the new iterate_over_types.
(dwflpp::iterate_over_types): Do the work that iterate_over_globals
used to handle, without assuming that we're starting in a top CU.
Josh Stone [Tue, 23 Aug 2011 23:59:16 +0000 (16:59 -0700)]
Improve scope splitting in function specs
We now take care to only split "::" separators that are outside of any
template specification. So "std::foo<std::bar>" becomes only two
components, "std" and "foo<std::bar>".
* util.cxx (tokenize_cxx): Split on "::" only outside of templates.
* tapsets.cxx (dwarf_query::parse_function_spec): Use tokenize_cxx.
The reported stuck process bug (whether it's in the kernel or
in stap), is apparently triggered by concurrent modprobes. Change
the testsuite to avoid this practice, to wallpaper over the problem
temporarily.
Stan Cox [Fri, 16 Sep 2011 15:31:39 +0000 (11:31 -0400)]
Add process.plt to probe plt entries.
* dwflpp.cxx (iterate_over_plt): New to callback through plt entries.
* dwflpp.h: Likewise.
* tapsets.cxx (TOK_PLT): New.
(base_query::query_plt_callback, query_plt): New methods.
(base_query): New members has_plt, has_statement, plt_val.
(base_query::base_query): Set new members.
(dwarf_query::query_module_symtab): Use raw address for plt entries.
(dwarf_query::add_probe_point): Don't reloc the plt entry address.
(query_module): Call iterate_over_plt.
(plt_expanding_visitor): New to expand $$name.
(dwarf_derived_probe::register_patterns): Add new probe tags.
Mark Wielaard [Fri, 16 Sep 2011 12:33:06 +0000 (14:33 +0200)]
Split kregs and uregs in probe context.
Keep track of kernel registers and user registers for a probe context
separately in anticipation of k/uregs "recovery". This patch simply
replaces the regs field of common_probe_context.h with a kregs and uregs
field, plus changes all over to use one or the other to store or fetch
the appropriate registers.
William Cohen [Thu, 15 Sep 2011 18:25:36 +0000 (14:25 -0400)]
PR13101 Networking tapsets should have structured comments
Fix up the language for the default filter functions for ipmib, linuxmib,
and tcpmib. Also make the functions return sane values regardless of
the value of the op parameter.
PR6503: verify build-ids of dynamically arriving modules
With the horrible module_sect_attr* hack, the .note.gnu.build-id data
may now be verified.
* runtime/sym.c (_stp_kmodule_check): New function.
* runtime/transport/symbols.c (_stp_module_notifier): Upon MODULE_STATE_COMING,
verify build-ids. Reject the module from further consideration upon a
mismatch.
PR6503: get at dynamic module section addresses via horrible hack (tm)
Since <linux/module.h> does not declare structs module_sect_attr[s],
let's declare it ourselves. We need to get at these, because
otherwise there is no way to get at the build-id section address.
* runtime/autoconf-module-sect-attrs.c: New (partial) autoconf file.
* buildrun.cxx (compile_pass): Build it.
* runtime/transport/symbols.c (_stp_module_notifier): Use our own
modern (2.6.19+) declaration of these structs to pass all section
names/addresses to _stp_kmodule_update_address.
Dave Brolley [Wed, 14 Sep 2011 20:21:01 +0000 (16:21 -0400)]
Systemtap compile-server and client do not require avahi.
Avahi is required for automatic detection and selection of compile-servers
but the client can still contact servers directly without avahi.
avahi-tools and avahi-devel are not required.
Updated stap-start-server, testsuite/lib/systemtap.exp and systemtap.spec
to reflect this. The specfile stills list avahi-client as a build
requirement for both the systemtap and systemtap-server packages.
Chris Meek [Wed, 14 Sep 2011 17:30:52 +0000 (13:30 -0400)]
PR13156: More verbose hash log output
Added descriptions for each item added to the script's hash in
the hash's log. Also, now it limits the length of each string
printed to the last 80 characters (except paths), to make it
more readable.
* runtime/transport/relay_v2.c (_stp_transport_data_fs_init):
In BULK_MODE, stop overestimating the number of pages we will
need: it's num_online_cpus rather than num_possible_cpus.
Chris Meek [Tue, 13 Sep 2011 17:48:50 +0000 (13:48 -0400)]
PR12221: Added Test Cases
Test:
1. Compiles a simple binary
2. Sets timestamp to a known value
3. Compiles a stap module probing this binary
4. Compiles a similar binary, same size, name, path etc
but different contents
5. Sets the timestamp to the same as the previous one
6. Compiles another stap module probing this new binary.
Results:
Before the build ID was added to the hash, this sequence
would cause stap to fetch the module from the cache, but
now, it will compile a new one, since the build IDs are
different.
Dave Brolley [Tue, 13 Sep 2011 15:07:11 +0000 (11:07 -0400)]
RHBZ 737095: Unable to start a systemtap server
- Don't use getlogin to obtain user name because it can fail.
- Establish server log before issuing any messages.
- Failure of avahi to advertise is now only a warning. Avahi is not
always available and the server can still be used directly.