Josh Stone [Fri, 21 May 2010 22:32:10 +0000 (15:32 -0700)]
Quiet the "checking uprobes" pass-4 messages
For session.verbose=1, we mostly only print the result of the pass. We
had a uprobes "preamble" though in pass 4, which IMO was a misleading
break in convention. It gave the impression that all the building time
for the actual script was part of the uprobes work.
* buildrun.cxx (verify_uprobes_uptodate, make_uprobes): Only print the
preamble message for verbose > 1.
Josh Stone [Thu, 20 May 2010 01:02:04 +0000 (18:02 -0700)]
PR10025: Don't read ahead tokens within preprocessor blocks
Since some aspects of the lexer are contextual, like wildcards in
probepoint identifiers, we need to wait for the right context in the
preprocessor rather than reading a bunch of tokens at once.
* parse.h (parser): Replace enqueued_pp with a pp_state stack.
* parse.cxx (parser::scan_pp): Use pp_state to decide if the current
token should be kept or skipped, and transition states as the various
%( %? %: %) are encountered.
(parser::skip_pp): Helper to leap over skipped tokens.
David Smith [Tue, 18 May 2010 18:19:23 +0000 (13:19 -0500)]
Fixed BZ 592830 by fixing signal.check_ignored.return for newer kernels.
* tapset/signal.stp (signal.check_ignored.return): Provide an alternative
for newer kernels where the sig_ignored() function has been inlined.
* tapset/errno.stp (return_str): New function.
* testsuite/buildok/signal-detailed.stp: Made the
signal.check_ignored.return test non-optional.
David Smith [Mon, 17 May 2010 20:45:32 +0000 (15:45 -0500)]
Added RHEL4 support to socket.stp and more testing.
* tapset/socket.stp: Removed kernel version checks and now works on RHEL4.
Tested on RHEL4, RHEL4, RHEL6(beta), f11, f12, and rawhide.
* testsuite/buildok/socket-detailed.stp: Renamed from socket.stp and updated.
* testsuite/buildok/socket-embedded.stp: Updated.
David Smith [Fri, 14 May 2010 18:03:53 +0000 (13:03 -0500)]
Update the ioblock and scheduler tapsets and testcases.
* tapset/ioblock.stp: Fixed several problems where the ioblock_trace
probes didn't all export the same convenience variables. Works on
RHEL4, RHEL5, RHEL6(beta), f11, f12, and rawhide.
* tapset/scheduler.stp: Fixed several problems where the kprobes backup
probes for tracepoints didn't work. Works on RHEL4, RHEL5, RHEL6(beta),
f11, f12, and rawhide.
* testsuite/buildok/ioblock-all-probes.stp: New testcase.
* testsuite/buildok/ioblock-detailed.stp: New detailed test of all ioblock
probes and convenience variables.
* testsuite/buildok/ioblock-embedded.stp: Added bio_rw_str() test.
* testsuite/buildok/ioblock_test.stp: Deleted old ioblock test.
* testsuite/buildok/scheduler-embedded.stp: New testcase.
* testsuite/buildok/scheduler-detailed.stp: New detailed test of all
scheduler probes and convenience variables.
* testsuite/buildok/scheduler-cpu_off.stp: New detailed test which can
fail on some platforms because we can't find inline function arguments.
* testsuite/buildok/scheduler-ctxswitch.stp: Ditto.
* testsuite/buildok/scheduler-migrate.stp: Ditto.
* testsuite/buildok/scheduler-test-tracepoints.stp: Deleted old test.
* testsuite/buildok/sched_test.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail the scheduler
single-probe tests.
Josh Stone [Thu, 13 May 2010 22:01:13 +0000 (15:01 -0700)]
PR11598: Allow "&" address-of in @defined()
* parse.cxx (parser::parse_target_symbol): Do addressof parsing here.
(parser::parse_value): Yield addressof parsing to parse_target_symbol.
(parser::parse_defined_op): Don't insist on an identifier, could be &.
* testsuite/semok/thirtysix.stp: Add tests for @defined(&...).
David Smith [Wed, 12 May 2010 17:49:03 +0000 (12:49 -0500)]
PR 9871 (partial) fix. Removed some embedded-C in context.stp and task.stp.
* tapset/context.stp: Converted several functions from embedded-C to
script language.
* tapset/task.stp (task_parent): Converted from embedded-C to script
languange.
* testsuite/buildok/context-embedded.stp: Moved function tests not
actually present in context.stp to their own files.
* testsuite/buildok/context-symbols-embedded.stp: New testcase (includes
some tests from context-embedded.stp).
* testsuite/buildok/context-unwind-embedded.stp: Ditto.
* testsuite/buildok/ucontext-symbols-embedded.stp: Ditto.
* testsuite/buildok/ucontext-unwind-embedded.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfails the ucontext tests if
the system doesn't support uprobes.
Pretending that pre/postfix ++/-- operations only writes to its argument
messes up the optimization of a variable that is assigned and then
only used through post/prefix ++/--operators.
David Smith [Tue, 11 May 2010 20:05:14 +0000 (15:05 -0500)]
Updated networking.stp and added full testing.
* tapset/networking.stp: For RHEL4, marked netdev.hard_transmit and
netdev.change_mac probes as optional.
* testsuite/buildok/netdev.stp: Deleted old networking test.
* testsuite/buildok/networking.stp: Ditto.
* testsuite/buildok/networking-all-probes.stp: New basic test of
networking.stp.
* testsuite/buildok/networking-embedded.stp: New test of functions in
networking.stp.
* testsuite/buildok/networking-detailed.stp: New detailed test of all
netdev probes and convenience variables.
* testsuite/buildok/networking-change_rx_flag.stp: New detailed test which
can fail on some platforms because we can't find inline function
arguments.
David Smith [Tue, 11 May 2010 17:58:17 +0000 (12:58 -0500)]
PR 9871 (partial) fix. Removed all remaining embedded-C in vfs.stp.
* tapset/vfs.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f11, f12, and rawhide. Removed all remaining embedded-C.
* testsuite/buildok/vfs-embedded.stp: Updated with more functions.
* testsuite/buildok/vfs-detailed.stp: New detailed test of all vfs probes
and convenience variables.
* testsuite/buildok/testsuite/buildok/vfs_testcase.stp: Deleted (replaced
by vfs-detailed.stp).
David Smith [Mon, 10 May 2010 20:32:28 +0000 (15:32 -0500)]
Updated ioscheduler.stp and tests.
* tapset/ioscheduler.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f11, f12, and rawhide.
* testsuite/buildok/ioscheduler-all-probes.stp: New test.
* testsuite/buildok/ioscheduler-detailed.stp: New detailed test of all
ioscheduler probes and convenience variables.
* testsuite/buildok/ioscheduler.stp: Deleted, since no longer needed with
the addition of ioscheduler-detailed.stp.
Josh Stone [Fri, 7 May 2010 19:07:26 +0000 (12:07 -0700)]
PR11266: Catch target_symbols as non-assignment lvalues
There are more ways to modify values than just assignment;
"struct visitor" knows this, but var_expanding_visitor needs
a lesson.
* tapsets.cxx (var_expanding_visitor::rewrite_lvalue): New, factored out
of visit_assignment to change various lvalues into function calls.
(var_expanding_visitor::visit_assignment): Use rewrite_lvalue.
(var_expanding_visitor::visit_pre_crement): New.
(var_expanding_visitor::visit_post_crement): New.
(var_expanding_visitor::visit_delete_statement): New.
* testsuite/semko/target_lvalue.stp: Test for exit-1 without crashing.
On jistone's advice, tested the code on rhel4 where there is no
utrace. print_ubacktrace() works only on incoming kernel state
(which shouldn't really work anyway). Forcing uprobes for these
ubacktrace() type calls is therefore not a problem.
* buildrun.cxx (uprobes_pass): If uprobes still needed, probably
due to /* pragma:uprobes */, rejectfor utrace-less kernels.
PR11573: infer need for uprobes from special embedded-c markup tag
* tapset/ucontext-unwind.stp: Mark up these babies with /* pragma:uprobes */.
* elaborate.cxx (typeresolution_info::visit_embeddedcode): Look for that
and activate session.need_uprobes if found.
docs: clean up stapfuncs/stapprobes man pages, deprecate probefunc()
* probefuncs.3stap.in: Remove most sections that are redundant with
automatically extracted documentation in individual function::*(stap)
man pages.
* probeprobes.3stap.in: Ditto for probes::*(stap).
* tapset/context-symbols.stp (probefunc): Describe as deprecated.
Change default timeout for global variable locks to apprx. 1 ms,
in 10us chunks. TRYLOCKDELAY now measures microseconds.
* translate.cxx (translate_pass): Change TRYLOCKDELAY and
MAXTRYLOCK default values.
* runtime/probe_lock.h (stp_lock_probe): Use udelay() rather than
ndelay().
* runtime/time.c (STP_TIMELOCK*): New macros just for time seqlock
timeout purposes. Expected contention is negligible here.
Josh Stone [Wed, 5 May 2010 23:47:19 +0000 (16:47 -0700)]
PR11556: Support array indexing on @cast pointers
It's a bit of a special case, because we don't have a pointer DIE for
the named type in @cast expressions. However, we can read the element
size manually and still simulate an array access.
* loc2c.c (pointer_stride): New, factored out of array_stride.
(c_translate_array_pointer): New, performs the pointer math based on
the element size of the pointee type.
* dwflpp.cxx (dwflpp::translate_components): Take a starting component.
(dwflpp::literal_stmt_for_pointer): If the initial type is not already
a pointer or array, we can fake an array access on the input pointer.
* testsuite/semok/cast.stp: Add an array access test on a @cast.
* testsuite/systemtap.base/cast.stp: Ditto.
* testsuite/systemtap.base/cast.exp: Ditto.
Josh Stone [Wed, 5 May 2010 22:52:55 +0000 (15:52 -0700)]
Straighten out die/attr use in dwflpp translation
It was confusing (to me) what was the expected state of each die_mem and
attr_mem passed around in dwflpp translation. I've changed it now so
that we pass just a vardie and typedie, which I think is clearer. This
is also nicer for @cast where we don't have any attr for the initial
type.
* dwflpp.cxx (dwarf_die_type): Helper to get the type, throw on error.
(dwflpp::translate_components): Just update a vardie and typedie.
(dwflpp::translate_final_fetch_or_store): Use a vardie and typedie.
(dwflpp::resolve_unqualified_inner_typedie): Update dies, not attrs.
(dwflpp::literal_stmt_for_local): Update calls to above.
(dwflpp::literal_stmt_for_return): Ditto.
(dwflpp::literal_stmt_for_pointer): Ditto.
David Smith [Wed, 5 May 2010 17:32:21 +0000 (12:32 -0500)]
PR 11338 (partial): Used @defined in tcp.stp and new test cases.
* tapset/tcp.stp: Removed kernel version check. Made other small
improvements.
* testsuite/buildok/tcp-detailed.stp: New detailed test of all tcp probes
and convenience variables.
* testsuite/buildok/udp-detailed.stp: New detailed test of all udp probes
and convenience variables.
* testsuite/buildok/tcp-embedded.stp: Added rest of the tcp embedded-C
functions.
David Smith [Tue, 4 May 2010 21:39:52 +0000 (16:39 -0500)]
Updated context tapset (for RHEL4) and updated the context testcase.
* tapset/context.stp: Added RHEL4 fixes.
* tapset/context-embedded.stp: Added rest of the context embedded-C
functions and renamed from context-test.stp.
* staptree.h: embedded_expr: New type.
* parse.cxx (parse_statement, parse_value): Parse it.
* staptree.cxx: Update visitors, provide basic code.
* elaborate.cxx (typeresolution_info::visit_embedded_expr): Select
type based upon internal markup.
* translate.cxx: Emit it.
* elaborate.h: Corresponding changes.
* NEWS, stap.1.in: Document it.
* testsuite/buildko/three.stp, buildok/thirtyfour.stp, parseok/twentyone.stp:
Test it.
David Smith [Tue, 4 May 2010 17:01:01 +0000 (12:01 -0500)]
PR 11338 (partial): Used @defined in signal.stp and general update.
* tapset/signal.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f12, and rawhide. Removed some kernel version checks.
* testsuite/buildok/signal-all-probes.stp: No longer tests "internal"
probes.
* testsuite/buildok/signal-detailed.stp: New detailed test of all signal
probes and convenience variables.
* testsuite/buildok/signal-check_ignored.stp: New detailed test which can
fail on some platforms because we can't find inline function arguments.
* testsuite/buildok/signal-handle.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail signal-check_ignored.stp
and signal-handle.stp
Dave Brolley [Fri, 30 Apr 2010 14:56:30 +0000 (10:56 -0400)]
PR 11441: Preparation for client/server merge into stap.
- Factor out main into initialization, argument parsing, argument checking
pass 0-4, pass 5, cleanup.
- Move systemtap_session methods into their own file (session.cxx)
PR11504 part: remove synthetic probe_NNNN name from error messages and most diagnostics
* elaborate.cxx (typeresolution_info::visit_target_symbol): Use
sole_location as opposed to derived_probe name.
(unresolved, invalid, mismatch): Remove the name/etc. prefix, as token
context is available.
* translate.cxx (emit_lock_decls): Print sole_location here too.
David Smith [Mon, 3 May 2010 18:52:54 +0000 (13:52 -0500)]
Move failing nfs test to its own testcase.
* testsuite/buildok/nfs-detailed.stp: Move nfs.fop.check_flags test to
nfs-fop.check_flags.stp (since it fails on RHEL5 and would mask other
nfs test failures).
* testsuite/buildok/nfs-fop.check_flags.stp: New test.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail nfs-fop.check_flags.stp.
David Smith [Mon, 3 May 2010 18:18:53 +0000 (13:18 -0500)]
PR 11338 (partial): Used @defined in memory.stp and general update.
* tapset/memory.stp: Large update to get working on RHEL4, RHEL5,
RHEL6(beta), f12, and rawhide. Removed all kernel version checks.
* testsuite/buildok/memory-detailed.stp: New detailed test of all memory
probes and convenience variables.
* testsuite/buildok/vm.tracepoints.kmalloc.stp: Moved into
memory-detailed.stp.
* testsuite/buildok/vm.tracepoints.stp: Ditto.
* testsuite/buildok/memory-mmap.stp: New detailed test which can fail on
some platforms because we can't find inline function arguments.
* testsuite/buildok/memory-write_shared_copy.stp: Ditto.
* testsuite/systemtap.pass1-4/buildok.exp: Kfail memory-mmap.stp and
memory-write_shared_copy.stp.
David Smith [Thu, 29 Apr 2010 17:58:01 +0000 (12:58 -0500)]
Get memory.stp embedded-C functions working on 2.6.9 kernels (RHEL4).
* tapset/memory.stp: Define __GFP_DMA32, GFP_DMA32, __GFP_ZERO,
__GFP_NOMEMALLOC, and __GFP_HARDWALL if not defined.
* testsuite/buildok/memory-embedded.stp: New testcase.
David Smith [Tue, 27 Apr 2010 16:18:58 +0000 (11:18 -0500)]
PR 11338 (partial): Used @defined in nfs.stp and general update.
* tapset/nfs.stp: Large update to get working on RHEL4, f12, and
rawhide. RHEL5 fails because of PR 1155 (can't find inline function
arguments). with probe nfs.fop.check_flags. Removed all kernel version
checks.
* testsuite/buildok/nfs-detailed.stp: New detailed test of all nfs probes
and convenience variables.
Josh Stone [Tue, 27 Apr 2010 01:16:51 +0000 (18:16 -0700)]
PR11340: Use local vardecls for tracepoint/marker args
This gets rid of the "probe_context_var" field of target_symbols, which
was a hacky way to resolve args into a special local variable. Now we
create a real local vardecl and mark it with "skip_init" so we know that
it will be filled in by derived_probe-specific code.
* staptree.h (target_symbol): Remove probe_context_var.
(vardecl): Add skip_init.
* elaborate.h (derived_probe::emit_probe_context_vars): Remove, as such
vars are now emitted along with all the other locals.
* elaborate.cxx (semantic_pass_opt2): Make skip_init locals immune to
read/write analysis.
(const_folder::visit_defined_op): No more probe_context_var.
(typeresolution_info::visit_symbol): Ditto.
* tapset-mark.cxx (mark_var_expanding_visitor::visit_target_symbol_arg):
Refer to the local name as a normal symbol reference.
(mark_derived_probe::mark_derived_probe): If target_symbols were used,
add vardecls to the probe locals.
(mark_derived_probe::emit_probe_context_vars): Remove.
* tapsets.cxx (var_expanding_visitor::visit_defined_op): For @defined,
there's no longer a "success" via probe_context_var.
(tracepoint_var_expanding_visitor::visit_target_symbol_arg): Refer to
the local name as a normal symbol reference.
(tracepoint_derived_probe::tracepoint_derived_probe): For each used
arg, add a vardecl to the probe locals.
(tracepoint_derived_probe::emit_probe_context_vars): Remove.
* translate.cxx (c_unparser::emit_common_header): No probe_context_vars.
(c_unparser::emit_probe): Don't init the skip_init locals.
(c_unparser_assignment::visit_symbol): No more probe_context_var.
David Smith [Mon, 26 Apr 2010 16:56:29 +0000 (11:56 -0500)]
PR 11338 (partial): Used @defined in nfsd.stp and general update.
* tapset/nfsd.stp: Large update to get working on RHEL4, RHEL5, f12, and
rawhide. Removed some embedded-C and kernel version checks.
* testsuite/buildok/nfsd-embedded.stp: Added the rest of the nfsd embedded
functions.
* testsuite/buildok/nfsd-detailed.stp: New detailed test of all nfsd
probes and convenience variables.
Josh Stone [Thu, 22 Apr 2010 01:53:03 +0000 (18:53 -0700)]
Consolidate load_aggregate logic
* translate.cxx (c_unparser::load_aggregate): Every caller needed to
figure out pre_agg and var vs. mapvar, so let's do it for them.
(c_unparser::visit_foreach_loop): Defer more to load_aggregate.
(c_unparser::visit_arrayindex): Ditto.
(c_unparser::visit_print_format): Ditto.
(c_unparser::visit_stat_op): Ditto.
(c_tmpcounter::load_aggregate): New, determine all the tmps needed to
mirror c_unparser::load_aggregate.
(c_tmpcounter::visit_foreach_loop): Call load_aggregate.
(c_tmpcounter::visit_arrayindex): Ditto.
(c_tmpcounter::visit_print_format): Ditto.
(c_tmpcounter::visit_stat_op): Ditto.
Josh Stone [Thu, 22 Apr 2010 00:46:38 +0000 (17:46 -0700)]
Let hist buckets be printed with other arguments
We already have quite a kludge to fall back from the print(@hist(...))
special case if the @hist happens to have a bucket arrayindex on it. In
that case we're printing scalars, so we can allow the print to contain
other scalar values too.
As noted in a comment, this still fails if the arrayindex is the start
of a larger expression. :(
* parse.cxx (parser::parse_symbol): Parse other print args after @hist.
* testsuite/buildok/print_histogram_entry.stp: Add a few prints to
exercise this (and one that still doesn't work).
Josh Stone [Thu, 22 Apr 2010 00:24:08 +0000 (17:24 -0700)]
PR2168: Support foreach over pmap histograms
* translate.cxx (c_tmpcounter::visit_foreach_loop): Add tmps for hist
indexes, only if it's not already cached from an outer foreach.
(c_unparser::visit_foreach_loop): Allow the possibility that the hist
var might need to be a mapvar.
* testsuite/buildok/pmap_foreach.stp: Add a nested foreach loop.
* testsuite/systemtap.base/foreach_value.stp: Add a nested foreach loop.
* testsuite/systemtap.base/foreach_value.exp: Add expected output.
Josh Stone [Wed, 21 Apr 2010 22:11:28 +0000 (15:11 -0700)]
Block local stats and arrays in pass 2
Parts of the translator get confused if stats or arrays are allowed
through as locals. It's easier to block it in pass 2 as soon as we can
detect this issue.
* elaborate.cxx (typeresolution_info::check_local): Make sure that probe
and function locals are scalar longs or strings.
(semantic_pass_types): Call check_local for functions and probes.
(symresolution_info::visit_symbol): Make auto-locals scalar.
* testsuite/semko/local_array.stp: Test that local arrays are rejected.
* testsuite/semko/local_stat.stp: Test that local stats are rejected.
* testsuite/transko/array01.stp: Move to semko.
David Smith [Thu, 22 Apr 2010 18:40:09 +0000 (13:40 -0500)]
Fixed nfsderrno.stp for RHEL4 (and added a test).
* tapset/nfsderrno.stp: Switched from using cpu_to_be32() to
__constant_htonl() to add RHEL4 support.
* testsuite/buildok/nfsderrno-embedded.stp: New test of nfsderrno.stp.
Mark Wielaard [Wed, 21 Apr 2010 09:58:09 +0000 (11:58 +0200)]
Add some kallsyms_on_each_symbol micro-optimisations.
These only help when your kprobe group consists of a list of symbols
that can all be resolved.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_init): Calculate
how many probe symbols need to be resolved. Pass to kprobe_resolve.
(kprobe_derived_probe_group::emit_module_decls): In kprobe_resolve
keep track of how many kprobe symbols haven't resolved yet. Return
-1 when all done.
Mark Wielaard [Wed, 21 Apr 2010 09:21:07 +0000 (11:21 +0200)]
PR11514 Only use kallsyms_on_each_symbol speedup when available.
* runtime/autoconf-kallsyms-on-each-symbol.c: New check.
* buildrun.cxx (compile_pass): Add new check to output_autoconf.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_decls): Only declare
kprobe_resolve when STAPCONF_KALLSYMS_ON_EACH_SYMBOL defined.
(kprobe_derived_probe_group::emit_module_init): Check whether to invoke
kallsyms_on_each_symbol optimization.
Mark Wielaard [Tue, 20 Apr 2010 21:03:30 +0000 (23:03 +0200)]
Warn if kprobe symbol name not found on non-optional probe.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_init): Call
_stp_warn when symbol name couldn't be resolved to an address and
the probe isn't optional like badkprobe.exp expects.
Mark Wielaard [Tue, 20 Apr 2010 20:06:15 +0000 (22:06 +0200)]
PR11514 Only expand kallsyms once for kprobe_derived_probe_group.
kprobe_register is really expensive when used for multiple symbol
based probes since it will call kallsyms_expand_symbol over and over
to uncompress the kernel symbol table. So flip the comparison loop
inside out by looping once over the kernel symbol table.
* tapsets.cxx (kprobe_derived_probe_group::emit_module_decls): Declare
kprobe_resolve helper function.
(kprobe_derived_probe_group::emit_module_init): Call kprobe_resolve
through kallsyms_on_each_symbol. Never register through symbol name,
just check that address has been filled in by kprobe_resolve.
David Smith [Tue, 20 Apr 2010 18:10:46 +0000 (13:10 -0500)]
Fixed PR 3833 by updating nfs_proc.stp (and removing kernel version checks).
* tapset/nfs_proc.stp: Large update to get working on RHEL4, RHEL5, f12,
and rawhide.
* testsuite/buildok/nfs_proc-detailed.stp: New detailed test of all
nfs_proc probes and convenience variables.