Josh Stone [Wed, 5 May 2010 22:52:55 +0000 (15:52 -0700)]
Straighten out die/attr use in dwflpp translation
It was confusing (to me) what was the expected state of each die_mem and
attr_mem passed around in dwflpp translation. I've changed it now so
that we pass just a vardie and typedie, which I think is clearer. This
is also nicer for @cast where we don't have any attr for the initial
type.
* dwflpp.cxx (dwarf_die_type): Helper to get the type, throw on error.
(dwflpp::translate_components): Just update a vardie and typedie.
(dwflpp::translate_final_fetch_or_store): Use a vardie and typedie.
(dwflpp::resolve_unqualified_inner_typedie): Update dies, not attrs.
(dwflpp::literal_stmt_for_local): Update calls to above.
(dwflpp::literal_stmt_for_return): Ditto.
(dwflpp::literal_stmt_for_pointer): Ditto.
David Smith [Wed, 5 May 2010 17:32:21 +0000 (12:32 -0500)]
PR 11338 (partial): Used @defined in tcp.stp and new test cases.
* tapset/tcp.stp: Removed kernel version check. Made other small
improvements.
* testsuite/buildok/tcp-detailed.stp: New detailed test of all tcp probes
and convenience variables.
* testsuite/buildok/udp-detailed.stp: New detailed test of all udp probes
and convenience variables.
* testsuite/buildok/tcp-embedded.stp: Added rest of the tcp embedded-C
functions.
David Smith [Tue, 4 May 2010 21:39:52 +0000 (16:39 -0500)]
Updated context tapset (for RHEL4) and updated the context testcase.
* tapset/context.stp: Added RHEL4 fixes.
* tapset/context-embedded.stp: Added rest of the context embedded-C
functions and renamed from context-test.stp.
* staptree.h: embedded_expr: New type.
* parse.cxx (parse_statement, parse_value): Parse it.
* staptree.cxx: Update visitors, provide basic code.
* elaborate.cxx (typeresolution_info::visit_embedded_expr): Select
type based upon internal markup.
* translate.cxx: Emit it.
* elaborate.h: Corresponding changes.
* NEWS, stap.1.in: Document it.
* testsuite/buildko/three.stp, buildok/thirtyfour.stp, parseok/twentyone.stp:
Test it.
David Smith [Tue, 4 May 2010 17:01:01 +0000 (12:01 -0500)]
PR 11338 (partial): Used @defined in signal.stp and general update.
* tapset/signal.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f12, and rawhide. Removed some kernel version checks.
* testsuite/buildok/signal-all-probes.stp: No longer tests "internal"
probes.
* testsuite/buildok/signal-detailed.stp: New detailed test of all signal
probes and convenience variables.
* testsuite/buildok/signal-check_ignored.stp: New detailed test which can
fail on some platforms because we can't find inline function arguments.
* testsuite/buildok/signal-handle.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail signal-check_ignored.stp
and signal-handle.stp
Dave Brolley [Fri, 30 Apr 2010 14:56:30 +0000 (10:56 -0400)]
PR 11441: Preparation for client/server merge into stap.
- Factor out main into initialization, argument parsing, argument checking
pass 0-4, pass 5, cleanup.
- Move systemtap_session methods into their own file (session.cxx)
PR11504 part: remove synthetic probe_NNNN name from error messages and most diagnostics
* elaborate.cxx (typeresolution_info::visit_target_symbol): Use
sole_location as opposed to derived_probe name.
(unresolved, invalid, mismatch): Remove the name/etc. prefix, as token
context is available.
* translate.cxx (emit_lock_decls): Print sole_location here too.
David Smith [Mon, 3 May 2010 18:52:54 +0000 (13:52 -0500)]
Move failing nfs test to its own testcase.
* testsuite/buildok/nfs-detailed.stp: Move nfs.fop.check_flags test to
nfs-fop.check_flags.stp (since it fails on RHEL5 and would mask other
nfs test failures).
* testsuite/buildok/nfs-fop.check_flags.stp: New test.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail nfs-fop.check_flags.stp.
David Smith [Mon, 3 May 2010 18:18:53 +0000 (13:18 -0500)]
PR 11338 (partial): Used @defined in memory.stp and general update.
* tapset/memory.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f12, and rawhide. Removed all kernel version checks.
* testsuite/buildok/memory-detailed.stp: New detailed test of all memory
probes and convenience variables.
* testsuite/buildok/vm.tracepoints.kmalloc.stp: Moved into
memory-detailed.stp.
* testsuite/buildok/vm.tracepoints.stp: Ditto.
* testsuite/buildok/memory-mmap.stp: New detailed test which can fail on
some platforms because we can't find inline function arguments.
* testsuite/buildok/memory-write_shared_copy.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail memory-mmap.stp and
memory-write_shared_copy.stp.
David Smith [Thu, 29 Apr 2010 17:58:01 +0000 (12:58 -0500)]
Get memory.stp embedded-C functions working on 2.6.9 kernels (RHEL4).
* tapset/memory.stp: Define __GFP_DMA32, GFP_DMA32, __GFP_ZERO,
__GFP_NOMEMALLOC, and __GFP_HARDWALL if not defined.
* testsuite/buildok/memory-embedded.stp: New testcase.
David Smith [Tue, 27 Apr 2010 16:18:58 +0000 (11:18 -0500)]
PR 11338 (partial): Used @defined in nfs.stp and general update.
* tapset/nfs.stp: Large update to get working on RHEL4, f12, and
rawhide. RHEL5 fails because of PR 1155 (can't find inline function
arguments). with probe nfs.fop.check_flags. Removed all kernel version
checks.
* testsuite/buildok/nfs-detailed.stp: New detailed test of all nfs probes
and convenience variables.
Josh Stone [Tue, 27 Apr 2010 01:16:51 +0000 (18:16 -0700)]
PR11340: Use local vardecls for tracepoint/marker args
This gets rid of the "probe_context_var" field of target_symbols, which
was a hacky way to resolve args into a special local variable. Now we
create a real local vardecl and mark it with "skip_init" so we know that
it will be filled in by derived_probe-specific code.
* staptree.h (target_symbol): Remove probe_context_var.
(vardecl): Add skip_init.
* elaborate.h (derived_probe::emit_probe_context_vars): Remove, as such
vars are now emitted along with all the other locals.
* elaborate.cxx (semantic_pass_opt2): Make skip_init locals immune to
read/write analysis.
(const_folder::visit_defined_op): No more probe_context_var.
(typeresolution_info::visit_symbol): Ditto.
* tapset-mark.cxx (mark_var_expanding_visitor::visit_target_symbol_arg):
Refer to the local name as a normal symbol reference.
(mark_derived_probe::mark_derived_probe): If target_symbols were used,
add vardecls to the probe locals.
(mark_derived_probe::emit_probe_context_vars): Remove.
* tapsets.cxx (var_expanding_visitor::visit_defined_op): For @defined,
there's no longer a "success" via probe_context_var.
(tracepoint_var_expanding_visitor::visit_target_symbol_arg): Refer to
the local name as a normal symbol reference.
(tracepoint_derived_probe::tracepoint_derived_probe): For each used
arg, add a vardecl to the probe locals.
(tracepoint_derived_probe::emit_probe_context_vars): Remove.
* translate.cxx (c_unparser::emit_common_header): No probe_context_vars.
(c_unparser::emit_probe): Don't init the skip_init locals.
(c_unparser_assignment::visit_symbol): No more probe_context_var.
David Smith [Mon, 26 Apr 2010 16:56:29 +0000 (11:56 -0500)]
PR 11338 (partial): Used @defined in nfsd.stp and general update.
* tapset/nfsd.stp: Large update to get working on RHEL4, RHEL5, f12, and
rawhide. Removed some embedded-C and kernel version checks.
* testsuite/buildok/nfsd-embedded.stp: Added the rest of the nfsd embedded
functions.
* testsuite/buildok/nfsd-detailed.stp: New detailed test of all nfsd
probes and convenience variables.
Josh Stone [Thu, 22 Apr 2010 01:53:03 +0000 (18:53 -0700)]
Consolidate load_aggregate logic
* translate.cxx (c_unparser::load_aggregate): Every caller needed to
figure out pre_agg and var vs. mapvar, so let's do it for them.
(c_unparser::visit_foreach_loop): Defer more to load_aggregate.
(c_unparser::visit_arrayindex): Ditto.
(c_unparser::visit_print_format): Ditto.
(c_unparser::visit_stat_op): Ditto.
(c_tmpcounter::load_aggregate): New, determine all the tmps needed to
mirror c_unparser::load_aggregate.
(c_tmpcounter::visit_foreach_loop): Call load_aggregate.
(c_tmpcounter::visit_arrayindex): Ditto.
(c_tmpcounter::visit_print_format): Ditto.
(c_tmpcounter::visit_stat_op): Ditto.
Josh Stone [Thu, 22 Apr 2010 00:46:38 +0000 (17:46 -0700)]
Let hist buckets be printed with other arguments
We already have quite a kludge to fall back from the print(@hist(...))
special case if the @hist happens to have a bucket arrayindex on it. In
that case we're printing scalars, so we can allow the print to contain
other scalar values too.
As noted in a comment, this still fails if the arrayindex is the start
of a larger expression. :(
* parse.cxx (parser::parse_symbol): Parse other print args after @hist.
* testsuite/buildok/print_histogram_entry.stp: Add a few prints to
exercise this (and one that still doesn't work).
Josh Stone [Thu, 22 Apr 2010 00:24:08 +0000 (17:24 -0700)]
PR2168: Support foreach over pmap histograms
* translate.cxx (c_tmpcounter::visit_foreach_loop): Add tmps for hist
indexes, only if it's not already cached from an outer foreach.
(c_unparser::visit_foreach_loop): Allow the possibility that the hist
var might need to be a mapvar.
* testsuite/buildok/pmap_foreach.stp: Add a nested foreach loop.
* testsuite/systemtap.base/foreach_value.stp: Add a nested foreach loop.
* testsuite/systemtap.base/foreach_value.exp: Add expected output.
Josh Stone [Wed, 21 Apr 2010 22:11:28 +0000 (15:11 -0700)]
Block local stats and arrays in pass 2
Parts of the translator get confused if stats or arrays are allowed
through as locals. It's easier to block it in pass 2 as soon as we can
detect this issue.
* elaborate.cxx (typeresolution_info::check_local): Make sure that probe
and function locals are scalar longs or strings.
(semantic_pass_types): Call check_local for functions and probes.
(symresolution_info::visit_symbol): Make auto-locals scalar.
* testsuite/semko/local_array.stp: Test that local arrays are rejected.
* testsuite/semko/local_stat.stp: Test that local stats are rejected.
* testsuite/transko/array01.stp: Move to semko.
David Smith [Thu, 22 Apr 2010 18:40:09 +0000 (13:40 -0500)]
Fixed nfsderrno.stp for RHEL4 (and added a test).
* tapset/nfsderrno.stp: Switched from using cpu_to_be32() to
__constant_htonl() to add RHEL4 support.
* testsuite/buildok/nfsderrno-embedded.stp: New test of nfsderrno.stp.
Mark Wielaard [Wed, 21 Apr 2010 09:58:09 +0000 (11:58 +0200)]
Add some kallsyms_on_each_symbol micro-optimisations.
These only help when your kprobe group consists of a list of symbols
that can all be resolved.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_init): Calculate
how many probe symbols need to be resolved. Pass to kprobe_resolve.
(kprobe_derived_probe_group::emit_module_decls): In kprobe_resolve
keep track of how many kprobe symbols haven't resolved yet. Return
-1 when all done.
Mark Wielaard [Wed, 21 Apr 2010 09:21:07 +0000 (11:21 +0200)]
PR11514 Only use kallsyms_on_each_symbol speedup when available.
* runtime/autoconf-kallsyms-on-each-symbol.c: New check.
* buildrun.cxx (compile_pass): Add new check to output_autoconf.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_decls): Only declare
kprobe_resolve when STAPCONF_KALLSYMS_ON_EACH_SYMBOL defined.
(kprobe_derived_probe_group::emit_module_init): Check whether to invoke
kallsyms_on_each_symbol optimization.
Mark Wielaard [Tue, 20 Apr 2010 21:03:30 +0000 (23:03 +0200)]
Warn if kprobe symbol name not found on non-optional probe.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_init): Call
_stp_warn when symbol name couldn't be resolved to an address and
the probe isn't optional like badkprobe.exp expects.
Mark Wielaard [Tue, 20 Apr 2010 20:06:15 +0000 (22:06 +0200)]
PR11514 Only expand kallsyms once for kprobe_derived_probe_group.
kprobe_register is really expensive when used for multiple symbol
based probes since it will call kallsyms_expand_symbol over and over
to uncompress the kernel symbol table. So flip the comparison loop
inside out by looping once over the kernel symbol table.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_decls): Declare
kprobe_resolve helper function.
(kprobe_derived_probe_group::emit_module_init): Call kprobe_resolve
through kallsyms_on_each_symbol. Never register through symbol name,
just check that address has been filled in by kprobe_resolve.
David Smith [Tue, 20 Apr 2010 18:10:46 +0000 (13:10 -0500)]
Fixed PR 3833 by updating nfs_proc.stp (and removing kernel version checks).
* tapset/nfs_proc.stp: Large update to get working on RHEL4, RHEL5, f12,
and rawhide.
* testsuite/buildok/nfs_proc-detailed.stp: New detailed test of all
nfs_proc probes and convenience variables.
Josh Stone [Tue, 20 Apr 2010 02:55:40 +0000 (19:55 -0700)]
PR11473: Document and test foreach "VALUE ="
* NEWS: Blurb it.
* stap.1.in: Describe it.
* testsuite/semko/foreachstmt03.stp: Test that stats die in pass-2.
* testsuite/systemtap.base/foreach_value.stp: Test with various types.
* testsuite/systemtap.base/foreach_value.exp: Driver for above.
* testsuite/systemtap.base/optim_arridx.exp: Include value in output.
Josh Stone [Tue, 20 Apr 2010 00:42:04 +0000 (17:42 -0700)]
PR11473: Add explicit iteration values with "VALUE ="
The foreach loops now have an optional "VALUE =" which assigns the
current iteration value to the named VALUE in each pass of the loop.
* staptree.h (foreach_loop::value): New field with the name to save.
* parse.cxx (parser::parse_foreach_loop): Parse "VALUE =", which
requires a lookahead to distinguish from an unbracketed "KEY in".
* staptree.cxx (foreach_loop::print): Display the value if present.
(*::visit_foreach_loop): Traverse the value too.
* elaborate.cxx (*::visit_foreach_loop): Ditto.
(add_global_var_display): Use "VALUE =" in generated code.
* translate.cxx (c_unparser::visit_foreach_loop): Assign the value in
the beginning of every foreach iteration.
Josh Stone [Tue, 20 Apr 2010 00:12:43 +0000 (17:12 -0700)]
PR11473: Optimize references to iteration values
When iterating with "foreach", we keep a map_node* which tells us our
current place in the map (and there's a similar marker for @hist). Most
of the time when users index a map within a foreach, it is to get the
current value using the keys given by foreach. When we know this is the
case, we can grab the value from the iterator instead of again searching
the map.
In the example given in PR11472, with two stat-map accesses within a
foreach, I recorded 12-15% improvement in runtime.
* translate.cxx (aggvar::get_hist): New accessor for indexing a @hist.
(mapvar::get_value): New accessor for reading a map iterator.
(mapvar::get_key): Simplify string NULL-check as done in get_value.
(c_unparser::visit_foreach_loop_value): New function to see if the
keys of a foreach are constant in the loop, and thus whether
references with those keys can be optimized.
(c_unparser::get_foreach_loop_value): Check for an active iteration
value which matches the given arrayindex.
(c_tmpcounter::visit_foreach_loop): Call visit_foreach_loop_value.
(c_unparser::visit_foreach_loop): Ditto.
(c_unparser::load_aggregate): Try to use a foreach value.
(c_unparser::visit_arrayindex): Ditto.
(c_tmpcounter::visit_arrayindex): Skip some tmps when foreach has it.
(c_tmpcounter::visit_print_format): Ditto.
(c_tmpcounter::visit_stat_op): Ditto.
Mark Wielaard [Mon, 19 Apr 2010 10:53:03 +0000 (12:53 +0200)]
Differentiate syscall and nd_syscall test results.
The systemtap.sum and .log files would use the same names for tests
run by the different nd_syscall.exp and syscall.exp testsuites. Make
sure they can be easily distinquished from each other.
* testsuite/systemtap.syscall/test.tcl (run_one_test): Take and use
extra suite parameter for test result reporting.
* testsuite/systemtap.syscall/syscall.exp (test_procedure): Provide
test suite name to run_one_test.
* testsuite/systemtap.syscall/nd_syscall.exp (test_procedure): Likewise.
Josh Stone [Fri, 16 Apr 2010 23:16:07 +0000 (16:16 -0700)]
Disallow --ldd in client options
Since --ldd invokes /usr/bin/ldd, it could be exploited by non-standard
PT_INTERP. By restricting it in client-options, we prevent stap-servers
from vulnerability.
PR11508: stap --ldd option for automagic shared library scanning
* session.h, main.cxx (unwindsym_ldd): New flag.
* translate.cxx (add_unwindsym_ldd): New function to perform ldd scan.
* NEWS, stap.1.in: Document it.
* testsuite/buildok/thirtythree.stp: Lightly test it.
David Smith [Fri, 16 Apr 2010 14:12:06 +0000 (09:12 -0500)]
Fixed PR 11480 by updating rpc.stp and its tests.
* tapset/rpc.stp: Large update to get working on RHEL4, RHEL5, f12, and
rawhide.
* testsuite/buildok/rpc-detailed.stp: New detailed test of all rpc probes.
* testsuite/buildok/rpc-embedded.stp: Updated.
* testsuite/systemtap.pass1-4/buildok.exp: Removed kfail for
rpc-all-probes.stp.
The former code allowed stapio to have two concurrent commands sent
toward the runtime at the same time: one from the main loop and
another one from the signal handling thread. This is suspected to
have caused occasional lockups. This new code switches to a deferred
signal handling model, where the signal handling thread merely
increments a counter, and the main loop monitors that count.
Unfortunately, this currently involves switching the main loop's
blocking /.ctl read into a nonblocking poll.
* runtime/staprun/mainloop.c (pending_interrupts): New variable.
(signal_thread): Increment it.
(stp_main_loop): Check it.
* main.cxx: Add "--compatible=VERSION" option.
* session.h (compatible): Store it.
* parse.cxx (eval_pp_conditional): Look at it as %( systemtap_v CMP VALUE ... %)
* stap.1.in: Document it.
* testsuite/parseko/preprocess17.stp, parseok/twenty.stp: Test it.
Josh Stone [Wed, 7 Apr 2010 23:41:41 +0000 (16:41 -0700)]
Clean up and clarify semantic_error chaining
I consolidated the copy-construction that every target_symbol::chain
caller was doing. I also removed the comments that target_symbol errors
might be chained due to function wildcards, because it's not true --
each derived instance gets a deep_copy of the probe body. However,
@cast can still chain since it may try multiple modules to resolve the
casting dereference.
David Smith [Wed, 7 Apr 2010 21:12:10 +0000 (16:12 -0500)]
PR 9871 (partial) fix. Removed embedded-C in ioblock.stp and ipmib.stp.
* tapset/ioblock.stp(__bio_start_sect): Converted from embedded-C to
script language.
* tapset/ipmib.stp: Replaced all _tcphdr_get_dport()/_tcphdr_get_sport()
calls with _tcp_skb_dport()/_tcp_skb_sport() calls. Removed embedded-C
functions _tcphdr_get_dport()/_tcphdr_get_sport().
(ipmib.OutRequests): Fixed for RHEL4.
(_trn_unspec): New function.
(_input_route_type): Originally called
__input_route_type_old()/__input_route_type_new(), which were embedded-C
functions. Now just does everything in script language.
(__input_route_type_old): Deleted.
(__input_route_type_new): Ditto.
(_output_route_type): Originally called
__output_route_type_old()/__output_route_type_new(), which were
embedded-C functions. Now just does everything in script language.
(__output_route_type_old): Deleted.
(__output_route_type_new): Ditto.
David Smith [Mon, 5 Apr 2010 19:54:45 +0000 (14:54 -0500)]
PR 9871 (partial) fix. Removed more embedded-C in nfs_proc.stp.
* tapset/nfs_proc.stp: Rewrote embedded-C functions stap_NFS_CLIENT(),
get_ip_from_client(), get_prot_from_client(), get_ip(), and get_prot()
in script language. Simplified __i2n_ip_proto() by just calling
get_ip_from_client() or get_prot_from_client().
Josh Stone [Tue, 30 Mar 2010 21:54:39 +0000 (14:54 -0700)]
Use a wider cache for simple function lookups
When we have many individual function lookups, like the nearly 1000 with
syscall.*, each one will iterate every CU in the module (M) and then do a
cache lookup in N entries. That's a thousand MlogN lookups.
We can instead keep the functions in a module-wide map, and then the
complexity is just a thousand logMN lookups.
* dwflpp.cxx (dwflpp::iterate_single_function): Do a simple function
lookup based on a module-wide cache.
(dwflpp::mod_function_caching_callback): Helper for above.
* tapsets.cxx (dwarf_query::query_module_functions): Query a single
function from the module-wide cache.
(dwarf_query::query_module_dwarf): Use above for simple cases.
Josh Stone [Tue, 30 Mar 2010 01:24:04 +0000 (18:24 -0700)]
Remove unused code from iterate_over_functions
* dwflpp.cxx (dwflpp::iterate_over_functions): No caller is using
has_statement_num anymore (since 6b517475), so kill it.
* tapsets.cxx (query_cu): Let the default call rule the day.
David Smith [Tue, 30 Mar 2010 20:53:51 +0000 (15:53 -0500)]
PR 9871 (partial) fix. Removed some embedded-C in ioblock.stp/vfs.stp.
* tapset/dev.stp: Added a bdevname() script function.
* tapset/ioblock.stp: Rewrote the embedded-C devname function to just use
bdevname() script function.
* tapset/vfs.stp: Removed embedded-C __bdevname() C function. Calls
bdevname() script function instead.
* tapset/string.stp: Added isdigit() function.