David Smith [Mon, 3 May 2010 18:52:54 +0000 (13:52 -0500)]
Move failing nfs test to its own testcase.
* testsuite/buildok/nfs-detailed.stp: Move nfs.fop.check_flags test to
nfs-fop.check_flags.stp (since it fails on RHEL5 and would mask other
nfs test failures).
* testsuite/buildok/nfs-fop.check_flags.stp: New test.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail nfs-fop.check_flags.stp.
David Smith [Mon, 3 May 2010 18:18:53 +0000 (13:18 -0500)]
PR 11338 (partial): Used @defined in memory.stp and general update.
* tapset/memory.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f12, and rawhide. Removed all kernel version checks.
* testsuite/buildok/memory-detailed.stp: New detailed test of all memory
probes and convenience variables.
* testsuite/buildok/vm.tracepoints.kmalloc.stp: Moved into
memory-detailed.stp.
* testsuite/buildok/vm.tracepoints.stp: Ditto.
* testsuite/buildok/memory-mmap.stp: New detailed test which can fail on
some platforms because we can't find inline function arguments.
* testsuite/buildok/memory-write_shared_copy.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail memory-mmap.stp and
memory-write_shared_copy.stp.
David Smith [Thu, 29 Apr 2010 17:58:01 +0000 (12:58 -0500)]
Get memory.stp embedded-C functions working on 2.6.9 kernels (RHEL4).
* tapset/memory.stp: Define __GFP_DMA32, GFP_DMA32, __GFP_ZERO,
__GFP_NOMEMALLOC, and __GFP_HARDWALL if not defined.
* testsuite/buildok/memory-embedded.stp: New testcase.
David Smith [Tue, 27 Apr 2010 16:18:58 +0000 (11:18 -0500)]
PR 11338 (partial): Used @defined in nfs.stp and general update.
* tapset/nfs.stp: Large update to get working on RHEL4, f12, and
rawhide. RHEL5 fails because of PR 1155 (can't find inline function
arguments). with probe nfs.fop.check_flags. Removed all kernel version
checks.
* testsuite/buildok/nfs-detailed.stp: New detailed test of all nfs probes
and convenience variables.
Josh Stone [Tue, 27 Apr 2010 01:16:51 +0000 (18:16 -0700)]
PR11340: Use local vardecls for tracepoint/marker args
This gets rid of the "probe_context_var" field of target_symbols, which
was a hacky way to resolve args into a special local variable. Now we
create a real local vardecl and mark it with "skip_init" so we know that
it will be filled in by derived_probe-specific code.
* staptree.h (target_symbol): Remove probe_context_var.
(vardecl): Add skip_init.
* elaborate.h (derived_probe::emit_probe_context_vars): Remove, as such
vars are now emitted along with all the other locals.
* elaborate.cxx (semantic_pass_opt2): Make skip_init locals immune to
read/write analysis.
(const_folder::visit_defined_op): No more probe_context_var.
(typeresolution_info::visit_symbol): Ditto.
* tapset-mark.cxx (mark_var_expanding_visitor::visit_target_symbol_arg):
Refer to the local name as a normal symbol reference.
(mark_derived_probe::mark_derived_probe): If target_symbols were used,
add vardecls to the probe locals.
(mark_derived_probe::emit_probe_context_vars): Remove.
* tapsets.cxx (var_expanding_visitor::visit_defined_op): For @defined,
there's no longer a "success" via probe_context_var.
(tracepoint_var_expanding_visitor::visit_target_symbol_arg): Refer to
the local name as a normal symbol reference.
(tracepoint_derived_probe::tracepoint_derived_probe): For each used
arg, add a vardecl to the probe locals.
(tracepoint_derived_probe::emit_probe_context_vars): Remove.
* translate.cxx (c_unparser::emit_common_header): No probe_context_vars.
(c_unparser::emit_probe): Don't init the skip_init locals.
(c_unparser_assignment::visit_symbol): No more probe_context_var.
David Smith [Mon, 26 Apr 2010 16:56:29 +0000 (11:56 -0500)]
PR 11338 (partial): Used @defined in nfsd.stp and general update.
* tapset/nfsd.stp: Large update to get working on RHEL4, RHEL5, f12, and
rawhide. Removed some embedded-C and kernel version checks.
* testsuite/buildok/nfsd-embedded.stp: Added the rest of the nfsd embedded
functions.
* testsuite/buildok/nfsd-detailed.stp: New detailed test of all nfsd
probes and convenience variables.
Josh Stone [Thu, 22 Apr 2010 01:53:03 +0000 (18:53 -0700)]
Consolidate load_aggregate logic
* translate.cxx (c_unparser::load_aggregate): Every caller needed to
figure out pre_agg and var vs. mapvar, so let's do it for them.
(c_unparser::visit_foreach_loop): Defer more to load_aggregate.
(c_unparser::visit_arrayindex): Ditto.
(c_unparser::visit_print_format): Ditto.
(c_unparser::visit_stat_op): Ditto.
(c_tmpcounter::load_aggregate): New, determine all the tmps needed to
mirror c_unparser::load_aggregate.
(c_tmpcounter::visit_foreach_loop): Call load_aggregate.
(c_tmpcounter::visit_arrayindex): Ditto.
(c_tmpcounter::visit_print_format): Ditto.
(c_tmpcounter::visit_stat_op): Ditto.
Josh Stone [Thu, 22 Apr 2010 00:46:38 +0000 (17:46 -0700)]
Let hist buckets be printed with other arguments
We already have quite a kludge to fall back from the print(@hist(...))
special case if the @hist happens to have a bucket arrayindex on it. In
that case we're printing scalars, so we can allow the print to contain
other scalar values too.
As noted in a comment, this still fails if the arrayindex is the start
of a larger expression. :(
* parse.cxx (parser::parse_symbol): Parse other print args after @hist.
* testsuite/buildok/print_histogram_entry.stp: Add a few prints to
exercise this (and one that still doesn't work).
Josh Stone [Thu, 22 Apr 2010 00:24:08 +0000 (17:24 -0700)]
PR2168: Support foreach over pmap histograms
* translate.cxx (c_tmpcounter::visit_foreach_loop): Add tmps for hist
indexes, only if it's not already cached from an outer foreach.
(c_unparser::visit_foreach_loop): Allow the possibility that the hist
var might need to be a mapvar.
* testsuite/buildok/pmap_foreach.stp: Add a nested foreach loop.
* testsuite/systemtap.base/foreach_value.stp: Add a nested foreach loop.
* testsuite/systemtap.base/foreach_value.exp: Add expected output.
Josh Stone [Wed, 21 Apr 2010 22:11:28 +0000 (15:11 -0700)]
Block local stats and arrays in pass 2
Parts of the translator get confused if stats or arrays are allowed
through as locals. It's easier to block it in pass 2 as soon as we can
detect this issue.
* elaborate.cxx (typeresolution_info::check_local): Make sure that probe
and function locals are scalar longs or strings.
(semantic_pass_types): Call check_local for functions and probes.
(symresolution_info::visit_symbol): Make auto-locals scalar.
* testsuite/semko/local_array.stp: Test that local arrays are rejected.
* testsuite/semko/local_stat.stp: Test that local stats are rejected.
* testsuite/transko/array01.stp: Move to semko.
David Smith [Thu, 22 Apr 2010 18:40:09 +0000 (13:40 -0500)]
Fixed nfsderrno.stp for RHEL4 (and added a test).
* tapset/nfsderrno.stp: Switched from using cpu_to_be32() to
__constant_htonl() to add RHEL4 support.
* testsuite/buildok/nfsderrno-embedded.stp: New test of nfsderrno.stp.
Mark Wielaard [Wed, 21 Apr 2010 09:58:09 +0000 (11:58 +0200)]
Add some kallsyms_on_each_symbol micro-optimisations.
These only help when your kprobe group consists of a list of symbols
that can all be resolved.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_init): Calculate
how many probe symbols need to be resolved. Pass to kprobe_resolve.
(kprobe_derived_probe_group::emit_module_decls): In kprobe_resolve
keep track of how many kprobe symbols haven't resolved yet. Return
-1 when all done.
Mark Wielaard [Wed, 21 Apr 2010 09:21:07 +0000 (11:21 +0200)]
PR11514 Only use kallsyms_on_each_symbol speedup when available.
* runtime/autoconf-kallsyms-on-each-symbol.c: New check.
* buildrun.cxx (compile_pass): Add new check to output_autoconf.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_decls): Only declare
kprobe_resolve when STAPCONF_KALLSYMS_ON_EACH_SYMBOL defined.
(kprobe_derived_probe_group::emit_module_init): Check whether to invoke
kallsyms_on_each_symbol optimization.
Mark Wielaard [Tue, 20 Apr 2010 21:03:30 +0000 (23:03 +0200)]
Warn if kprobe symbol name not found on non-optional probe.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_init): Call
_stp_warn when symbol name couldn't be resolved to an address and
the probe isn't optional like badkprobe.exp expects.
Mark Wielaard [Tue, 20 Apr 2010 20:06:15 +0000 (22:06 +0200)]
PR11514 Only expand kallsyms once for kprobe_derived_probe_group.
kprobe_register is really expensive when used for multiple symbol
based probes since it will call kallsyms_expand_symbol over and over
to uncompress the kernel symbol table. So flip the comparison loop
inside out by looping once over the kernel symbol table.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_decls): Declare
kprobe_resolve helper function.
(kprobe_derived_probe_group::emit_module_init): Call kprobe_resolve
through kallsyms_on_each_symbol. Never register through symbol name,
just check that address has been filled in by kprobe_resolve.
David Smith [Tue, 20 Apr 2010 18:10:46 +0000 (13:10 -0500)]
Fixed PR 3833 by updating nfs_proc.stp (and removing kernel version checks).
* tapset/nfs_proc.stp: Large update to get working on RHEL4, RHEL5, f12,
and rawhide.
* testsuite/buildok/nfs_proc-detailed.stp: New detailed test of all
nfs_proc probes and convenience variables.
Josh Stone [Tue, 20 Apr 2010 02:55:40 +0000 (19:55 -0700)]
PR11473: Document and test foreach "VALUE ="
* NEWS: Blurb it.
* stap.1.in: Describe it.
* testsuite/semko/foreachstmt03.stp: Test that stats die in pass-2.
* testsuite/systemtap.base/foreach_value.stp: Test with various types.
* testsuite/systemtap.base/foreach_value.exp: Driver for above.
* testsuite/systemtap.base/optim_arridx.exp: Include value in output.
Josh Stone [Tue, 20 Apr 2010 00:42:04 +0000 (17:42 -0700)]
PR11473: Add explicit iteration values with "VALUE ="
The foreach loops now have an optional "VALUE =" which assigns the
current iteration value to the named VALUE in each pass of the loop.
* staptree.h (foreach_loop::value): New field with the name to save.
* parse.cxx (parser::parse_foreach_loop): Parse "VALUE =", which
requires a lookahead to distinguish from an unbracketed "KEY in".
* staptree.cxx (foreach_loop::print): Display the value if present.
(*::visit_foreach_loop): Traverse the value too.
* elaborate.cxx (*::visit_foreach_loop): Ditto.
(add_global_var_display): Use "VALUE =" in generated code.
* translate.cxx (c_unparser::visit_foreach_loop): Assign the value in
the beginning of every foreach iteration.
Josh Stone [Tue, 20 Apr 2010 00:12:43 +0000 (17:12 -0700)]
PR11473: Optimize references to iteration values
When iterating with "foreach", we keep a map_node* which tells us our
current place in the map (and there's a similar marker for @hist). Most
of the time when users index a map within a foreach, it is to get the
current value using the keys given by foreach. When we know this is the
case, we can grab the value from the iterator instead of again searching
the map.
In the example given in PR11472, with two stat-map accesses within a
foreach, I recorded 12-15% improvement in runtime.
* translate.cxx (aggvar::get_hist): New accessor for indexing a @hist.
(mapvar::get_value): New accessor for reading a map iterator.
(mapvar::get_key): Simplify string NULL-check as done in get_value.
(c_unparser::visit_foreach_loop_value): New function to see if the
keys of a foreach are constant in the loop, and thus whether
references with those keys can be optimized.
(c_unparser::get_foreach_loop_value): Check for an active iteration
value which matches the given arrayindex.
(c_tmpcounter::visit_foreach_loop): Call visit_foreach_loop_value.
(c_unparser::visit_foreach_loop): Ditto.
(c_unparser::load_aggregate): Try to use a foreach value.
(c_unparser::visit_arrayindex): Ditto.
(c_tmpcounter::visit_arrayindex): Skip some tmps when foreach has it.
(c_tmpcounter::visit_print_format): Ditto.
(c_tmpcounter::visit_stat_op): Ditto.
Mark Wielaard [Mon, 19 Apr 2010 10:53:03 +0000 (12:53 +0200)]
Differentiate syscall and nd_syscall test results.
The systemtap.sum and .log files would use the same names for tests
run by the different nd_syscall.exp and syscall.exp testsuites. Make
sure they can be easily distinquished from each other.
* testsuite/systemtap.syscall/test.tcl (run_one_test): Take and use
extra suite parameter for test result reporting.
* testsuite/systemtap.syscall/syscall.exp (test_procedure): Provide
test suite name to run_one_test.
* testsuite/systemtap.syscall/nd_syscall.exp (test_procedure): Likewise.
Josh Stone [Fri, 16 Apr 2010 23:16:07 +0000 (16:16 -0700)]
Disallow --ldd in client options
Since --ldd invokes /usr/bin/ldd, it could be exploited by non-standard
PT_INTERP. By restricting it in client-options, we prevent stap-servers
from vulnerability.
PR11508: stap --ldd option for automagic shared library scanning
* session.h, main.cxx (unwindsym_ldd): New flag.
* translate.cxx (add_unwindsym_ldd): New function to perform ldd scan.
* NEWS, stap.1.in: Document it.
* testsuite/buildok/thirtythree.stp: Lightly test it.
David Smith [Fri, 16 Apr 2010 14:12:06 +0000 (09:12 -0500)]
Fixed PR 11480 by updating rpc.stp and its tests.
* tapset/rpc.stp: Large update to get working on RHEL4, RHEL5, f12, and
rawhide.
* testsuite/buildok/rpc-detailed.stp: New detailed test of all rpc probes.
* testsuite/buildok/rpc-embedded.stp: Updated.
* testsuite/systemtap.pass1-4/buildok.exp: Removed kfail for
rpc-all-probes.stp.
The former code allowed stapio to have two concurrent commands sent
toward the runtime at the same time: one from the main loop and
another one from the signal handling thread. This is suspected to
have caused occasional lockups. This new code switches to a deferred
signal handling model, where the signal handling thread merely
increments a counter, and the main loop monitors that count.
Unfortunately, this currently involves switching the main loop's
blocking /.ctl read into a nonblocking poll.
* runtime/staprun/mainloop.c (pending_interrupts): New variable.
(signal_thread): Increment it.
(stp_main_loop): Check it.
* main.cxx: Add "--compatible=VERSION" option.
* session.h (compatible): Store it.
* parse.cxx (eval_pp_conditional): Look at it as %( systemtap_v CMP VALUE ... %)
* stap.1.in: Document it.
* testsuite/parseko/preprocess17.stp, parseok/twenty.stp: Test it.
Josh Stone [Wed, 7 Apr 2010 23:41:41 +0000 (16:41 -0700)]
Clean up and clarify semantic_error chaining
I consolidated the copy-construction that every target_symbol::chain
caller was doing. I also removed the comments that target_symbol errors
might be chained due to function wildcards, because it's not true --
each derived instance gets a deep_copy of the probe body. However,
@cast can still chain since it may try multiple modules to resolve the
casting dereference.
David Smith [Wed, 7 Apr 2010 21:12:10 +0000 (16:12 -0500)]
PR 9871 (partial) fix. Removed embedded-C in ioblock.stp and ipmib.stp.
* tapset/ioblock.stp(__bio_start_sect): Converted from embedded-C to
script language.
* tapset/ipmib.stp: Replaced all _tcphdr_get_dport()/_tcphdr_get_sport()
calls with _tcp_skb_dport()/_tcp_skb_sport() calls. Removed embedded-C
functions _tcphdr_get_dport()/_tcphdr_get_sport().
(ipmib.OutRequests): Fixed for RHEL4.
(_trn_unspec): New function.
(_input_route_type): Originally called
__input_route_type_old()/__input_route_type_new(), which were embedded-C
functions. Now just does everything in script language.
(__input_route_type_old): Deleted.
(__input_route_type_new): Ditto.
(_output_route_type): Originally called
__output_route_type_old()/__output_route_type_new(), which were
embedded-C functions. Now just does everything in script language.
(__output_route_type_old): Deleted.
(__output_route_type_new): Ditto.
David Smith [Mon, 5 Apr 2010 19:54:45 +0000 (14:54 -0500)]
PR 9871 (partial) fix. Removed more embedded-C in nfs_proc.stp.
* tapset/nfs_proc.stp: Rewrote embedded-C functions stap_NFS_CLIENT(),
get_ip_from_client(), get_prot_from_client(), get_ip(), and get_prot()
in script language. Simplified __i2n_ip_proto() by just calling
get_ip_from_client() or get_prot_from_client().
Josh Stone [Tue, 30 Mar 2010 21:54:39 +0000 (14:54 -0700)]
Use a wider cache for simple function lookups
When we have many individual function lookups, like the nearly 1000 with
syscall.*, each one will iterate every CU in the module (M) and then do a
cache lookup in N entries. That's a thousand MlogN lookups.
We can instead keep the functions in a module-wide map, and then the
complexity is just a thousand logMN lookups.
* dwflpp.cxx (dwflpp::iterate_single_function): Do a simple function
lookup based on a module-wide cache.
(dwflpp::mod_function_caching_callback): Helper for above.
* tapsets.cxx (dwarf_query::query_module_functions): Query a single
function from the module-wide cache.
(dwarf_query::query_module_dwarf): Use above for simple cases.
Josh Stone [Tue, 30 Mar 2010 01:24:04 +0000 (18:24 -0700)]
Remove unused code from iterate_over_functions
* dwflpp.cxx (dwflpp::iterate_over_functions): No caller is using
has_statement_num anymore (since 6b517475), so kill it.
* tapsets.cxx (query_cu): Let the default call rule the day.
David Smith [Tue, 30 Mar 2010 20:53:51 +0000 (15:53 -0500)]
PR 9871 (partial) fix. Removed some embedded-C in ioblock.stp/vfs.stp.
* tapset/dev.stp: Added a bdevname() script function.
* tapset/ioblock.stp: Rewrote the embedded-C devname function to just use
bdevname() script function.
* tapset/vfs.stp: Removed embedded-C __bdevname() C function. Calls
bdevname() script function instead.
* tapset/string.stp: Added isdigit() function.
Stan Cox [Fri, 26 Mar 2010 21:14:32 +0000 (17:14 -0400)]
Add kprobe user probe semaphore handling.
Adds stap_kprobe_process_found and stap_kprobe_mmap_found handlers
and then the task_finder uses those to set the semaphore values.
The probe adds user_path and user_lib which are used by the
task_finder to decide what object to sniff.
* sdt.h (STAP_SEMAPHORE): Also define for kprobes.
* tapsets.cxx (dwarf_derived_probe): Add user_path and user_lib.
(dwarf_derived_probe::join_group): Enable task finder for kprobes.
(dwarf_derived_probe::dwarf_derived_probe): Set user_path and
user_lib.
(dwarf_derived_probe_group::emit_module_decls): Emit
stap_kprobe_mmap_found, sdt_sem_offset, sdt_sem_address, pathname,
and finder declarations. Emit stap_kprobe_process_found and
stap_kprobe_mmap_found.
(dwarf_derived_probe_group::emit_module_init): Setup task finder.
(sdt_query::convert_location): Remove TOK_LIBRARY token.
(dwarf_builder::build): Set user_path and user_lib.
(kprobe_derived_probe): Add path and library.
(kprobe_builder::build): Get process and library params
David Smith [Fri, 26 Mar 2010 19:17:48 +0000 (14:17 -0500)]
PR 11338 (partial): Used '@defined()' in inet_sock tapsets.
* tapset/inet_sock.stp (inet_get_local_port): Used '@defined' instead of a
kernel version check. Added RHEL4 support.
(inet_get_ip_source): Now calls __ip_sock_daddr() to get daddr.
David Smith [Fri, 26 Mar 2010 18:24:38 +0000 (13:24 -0500)]
PR 9871/11338 fix in tcpmib.stp by removing embedded-C and using @defined.
* tapset/tcpmib.stp (tcpmib_get_state): Changed from embedded-C to script
code.
(tcpmib_local_addr): Instead of an embedded-C function, now calls
function from ip.stp/tcp.stp.
(tcpmib_remote_addr): Ditto.
(tcpmib_local_port): Ditto.
(tcpmib_remote_port): Ditto.
(tcpmib.OutRsts.A): Used '@defined' instead of a kernel version check.
* tapset/ip.stp (__ip_sock_saddr): Used '@defined' instead of a kernel
version check. Added RHEL4 support.
(__ip_sock_daddr): Ditto.
* tapset/tcp.stp (__tcp_sock_dport): Ditto.
(__tcp_sock_sport): Ditto.
Josh Stone [Fri, 26 Mar 2010 02:32:46 +0000 (19:32 -0700)]
Print probepoint addresses in hex
* staptree.h (literal_number): Add a bool "print_hex".
* staptree.cxx (literal_number::print): Switch to hex if needed.
* tapsets.cxx (dwarf_derived_probe::dwarf_derived_probe): Write the
statement number component with a hex literal_number.
(sdt_query::convert_location): Ditto
(kprobe_derived_probe::kprobe_derived_probe): Ditto.
(hwbkpt_derived_probe::hwbkpt_derived_probe): Ditto for data addr.
Josh Stone [Thu, 25 Mar 2010 21:31:45 +0000 (14:31 -0700)]
PR11399: Allow '**' to match tapsets across '.'
Normally wildcards are constrained to matching a single probe point
component between the '.' separator. This patch enables '**' to match
across the separator.
Probe point parameters are still absolute separators though, as there's
not really a meaningful semantic we could use to cross them with '**'.
* elaborate.cxx (isdoubleglob): Check for '**'.
(match_node::find_and_build): Recurse '**' to cross separators.
* testsuite/semok/doubleglob.stp: New test of broad wildcards.
* testsuite/semko/doubleglob.stp: New negative test.
David Smith [Wed, 24 Mar 2010 15:57:01 +0000 (10:57 -0500)]
Improved atomic.exp testcase by removing the need for an external module.
* testsuite/systemtap.base/atomic.exp: Removed the need for the external
module by putting the atomic_long_t inside the stap module.
* testsuite/systemtap.base/atomic_module.c: Removed.
* testsuite/systemtap.base/atomic_module.makefile: Ditto.
* testsuite/lib/stap_run_error.exp (stap_run_error): Better error handling.