David Smith [Thu, 20 Aug 2015 16:27:53 +0000 (11:27 -0500)]
Fix PR18856 by making nfsd.close optional.
* tapset/linux/nfsd.stp: Make nfsd.close optional in nfsd.entries (and add
it optionally to nfsd.return), since the underlying probe point no
longer exists in kernels 4.2+. Also add nfsd.proc.commit.return to
nfsd.proc.return (to match nfsd.proc).
* testsuite/buildok/nfsd-detailed.stp: Make nfsd.close tests optional.
Josh Stone [Thu, 20 Aug 2015 01:38:08 +0000 (18:38 -0700)]
runtime: only compare line records at "commit" states
The state machine for line records may go through intermediate states
which are not proper to compare address against. Only at DW_LNS_copy,
DW_LNE_end_sequence, and special opcodes are we supposed to "append a
row to the matrix", as the DWARF specification tells us.
The usymfileline testcase didn't hit problems, because its C target only
outputted a simple sequence of special opcodes. It's now C++, as that's
a very easy way to get code generated from a lot of different files and
lines. A simple std::find creates pretty complicated line records, even
in an unoptimized build.
Abegail Jakop [Wed, 19 Aug 2015 22:04:36 +0000 (18:04 -0400)]
stringtable.h: add logical ops and to_string if needed
when it comes to string_ref, boost 1.53 doesn't contain all the
logical operations that we need, nor to_string(). so we'll fill in
the gaps with our own defintions for interned_string.
Martin Cermak [Wed, 19 Aug 2015 18:20:36 +0000 (20:20 +0200)]
Update syscall tapset documentation to use cached content.
Before this update syscall tapset autodocumentation has been
generated dynamically at the build time using stap -L call.
This way stap requires kernel-devel RPM matching `uname -r`
which is tricky to achieve in koji. For this reason the content
is cached from now on:
jistone pointed out a possible problem in the interned_string
implementation; we break the tie in favour of trivial c_str()
(since commit 3e4547ad096d showed its value).
Abegail Jakop [Fri, 14 Aug 2015 20:42:33 +0000 (16:42 -0400)]
alternate interned_string if no boost::string_ref
string_ref was only introduced in boost 1.53, meaning rhel 5 and 6
might not have a recent enough version. to account for this, if
string_ref is not available, fall back on an alternate definition
of interned_string that uses std::string instead.
Some of the tapset*cxx functions were used to representing empty
function / file names with NULL char*s. As these are transitioned to
interned_strings, the latter's ctors must be willing to accept those
NULLs and convert them to empty strings. (Adjusting all the callers
to pass "" instead of NULL in those slots is also possible.)
interned_string: drop magic c_str() implementation
It turns out that while it was right for boost::string_ref not to
provide it, we can, because our interned_string's backing stores
are full (read-only) std::strings, which always carry a \0. This
shrinks down our interned_string to 2*sizeof(void*) bytes.
With the richer interned_string API, one can return to the previous
way of finding line boundaries in the input string. While in the
area, we tweak spacing of parse error message lines.
With INTERNED_STRING_INSTRUMENT set, a /tmp/hash.log file is written
out to record each string being interned. Looking through that is a
good way of tuning the hash function, as well as what
staptree/etc. variables are reverted to std::string form. (The hint
is that if strings are hardly ever interned in the log, then maybe
they are not reused either. But that could instead be because they're
being reused exclusively in interned string_ref form, so don't need to
be rehashed.)
Experimentation indicates that the sizeof(interned_string)=24
is so much larger than that of an empty sizeof(std::string)=8
that for mostly-empty slots it's a loss! A 1-byte std::string
causes a 26-byte malloc, so the interned_string is probably
good for nonempty strings.
Converting many many classes' use of std::string to interned_string.
Largely mechanical, and driven by the desire to (a) store long-lived
duplicate strings only once (in the stringtable), and (b) avoid
instantiating std::strings unnecessarily (in temporary variables
or function parameters).
Motivated by frustratingly easy-to-miss memory corruption errors of
the direct boost::string_ref model, switching gears to a new derived
class interned_string. This one performs proper interning when
converting in from a std::string for all copy/assignment operations.
In many cases, it can simply replace std::string through automatic
conversions in/out, naturally paying a CPU / temporary-memory cost.
Replacing many std::string types, focusing on those stored in
high-cardinality data structures, with boost::string_ref.
Largely mechanical work, with valgrind runs identifying some
persistent trouble-spots of the form
string_ref FOO = (some_temp_string_value);
which makes them dangle the moment the temp goes out of scope.
string_ref usage correction: drop string_ref->to_string()->c_str() chain
This construct generates stale char* pointers that valgrind notices.
We use a first-class temporary string object for these uses. (They
should probably be rewritten in terms of string_ref native
operations.)
The obstacks used for storing piecemeal-printf'd $context functions
could leak if inferior functions threw exceptions. We now use a
RAII type widget to assure their destruction.
It turns out that c++11's unordered_set<> makes the stringtable
dramatically faster. Since that is a c++11-only facility, we switch
the translator (stap) to conditionally compile with that flag.
Returning partly to an older model of parser input processing: the
parser::input_contents goes back to a plain string, which may be
modified gradually by $/@ expansion etc. While string_refs cannot be
made based on that, they don't need to be either, since the
input_contents is a temporary member variable and will be disposed of
when the parser dies.
So we switch back to the prior general logic of
lexer::input_{put,get,peek}, which worked OK. To populate token
content string_refs, we use a global stringtable (a *set<string>)
against which long-lived string_refs may be issued with a new intern()
call. The input file contents are amongst those strings interned.
(Many other repeated string uses in the translator could probably
benefit from intern/string_ref'ing, and there are optimization
opportunities within the stringtable implementation proper.)
(While en route, mass-dropped a bunch of unnecessary namespace
prefixes in some affected files. Sorry for the diff noise.)
parse::parse_probe_point(): store new token's content
in parse_porbe_point() the content for a new token is created by
appending to an existing token's content. the string for the new
token's content needs to be stored somewhere, so we'll store it in
the the probe_point, since we can't access the (private) stapfile.
store file contents as strings in systemtap_session
since we need input_contents and file_contents to last longer
than the lexer, store the input_contents in a vector in a
systemtap_session. change lexer->input_contents and
stapfile->file_contents to string* so they will point to part of
some string in session->input_contents. will need to change to
string_ref in another commit
parse.cxx: new append_to_content() and set_token_content() to hide
the logic of work of setting up a token's content.
staptree.h: a vector to contain all the mangled strings that tok->content
will point to
since tok->content are now boost::string_refs, some portions of the
code don't play well with the fact that string_refs can't be
automatically converted into a std::string, so we need to manually add
in a to_string() call to do the conversion.
Note: can't use .data() since the string_ref is proabably not null terminated.
session.*: store lex_cast() versions of the stap arguments passed
through the command line.
parse.cxx: if the scanner encounters [$|@]<NN> in a stap script, use
the pre-lex_cast() stap arguments instead of calling lex_cast() each
time the scanner encounters [$|@]<NN>.
parse.cxx: instead of keeping track of a seperate set of file_contents
(lexer->input_contents) in which [$|@]<NN> are replaced by script args,
we'll keep file_contents and input_contents identical to one another
and parse the scripts args strings when we encouter [$|@]<NN>
Felix Lu [Wed, 5 Aug 2015 17:55:53 +0000 (13:55 -0400)]
PR12151: Fix recursion problem, add stable synthetic var reading functions
elaborate.cxx - When checking for stable flag, does not follow
functioncall chains. All stable functions are now identified at beginning
of semantic_pass_opt_7.
tapsets.cxx - In non guru-mode, context variable reading functions
are cached. Moved optimization step 7 after duplicate functioncall
pass to remove duplicate synthetic functions first.
Felix Lu [Thu, 30 Jul 2015 20:09:05 +0000 (16:09 -0400)]
Add testcases for stable optimization
* testsuite/systemtap.base/stable* - Testing to make sure that
the result of optimized script matches nonoptimized.
Also checks that the embedded-c functions are called the correct
number of times.
Martin Cermak [Mon, 3 Aug 2015 13:24:24 +0000 (15:24 +0200)]
Update file mode for testsuite/buildok/stap_staticmarkers-detailed.stp
Missing the 'execute' perm on stap_staticmarkers-detailed.stp causes
`ERROR: tcl error sourcing ./systemtap.pass1-4/buildok.exp` during
make installcheck.
Martin Cermak [Fri, 31 Jul 2015 15:40:18 +0000 (17:40 +0200)]
Fix PR18711 by updating the netfilter code for new rhel-7.2 kernels.
* buildrun.cxx (compile_pass): Add new netfilter autoconf test.
* runtime/linux/autoconf-netfilter-313b.c: New autoconf test.
* tapset-netfilter.cxx: (emit_module_decls): Add support for new
netfilter code backported to kernel-3.10.0-284.el7.
Martin Cermak [Thu, 16 Jul 2015 14:33:31 +0000 (16:33 +0200)]
Update rename.c syscall test for 3.10.0-294.el7 kernels.
On ppc64le, 'renameat2' syscall is unimplemented as defined in the
arch/powerpc/include/asm/systbl.h file. This change also allows
'renameat2' syscall to succeed when renaming a directory to a file
in case RENAME_EXCHANGE is defined.
David Smith [Tue, 14 Jul 2015 21:19:11 +0000 (16:19 -0500)]
Update the [nd_]syscall.clone probe aliases for rawhide.
* tapset/linux/syscalls.stp: Add '_do_fork' as an alternate probe point
for syscall.fork. Kernel commit 3033f14ab78c32 renamed 'do_fork' to
'_do_fork'.
* tapset/linux/nd_syscalls.stp: Ditto.
Martin Cermak [Tue, 14 Jul 2015 06:05:50 +0000 (08:05 +0200)]
PR18630: New testcase for dwarfless parameters from a uprobe.
* tapset/s390/registers.stp: Remove constraint added by commit eefd579b blocking data acquistion from a uprobe. This is per
PR18649#c1 and subsequent IRC chat with dsmith, the author
of the constraint.
* testsuite/systemtap.base/uprobe_nd_params.exp: New testcase.
David Smith [Tue, 7 Jul 2015 19:01:27 +0000 (14:01 -0500)]
Fixed PR18634 by getting timer probes to compile on rawhide.
* runtime/linux/timer.c (_stp_hrtimer_init): If STAPCONF_HRTIMER_GET_RES
isn't defined, meaning hrtimer_get_res() doesn't exist, just use the
'hrtimer_resolution' variable.
* buildrun.cxx (compile_pass): Add an export test for hrtimer_get_res().
* translate.cxx (emit_common_header): Remove generated inclusion of
linux/hrtimer.h.
* runtime/linux/timer.h: Add inclusion of linux/hrtimer.h.
David Smith [Tue, 7 Jul 2015 15:41:43 +0000 (10:41 -0500)]
Fixed BZ1240682 by fixing a rawhide compile problem.
* translate.cxx (translate_pass): Move the inclusion of runtime.h up
before the declaration of the module refresh mutex, to avoid compilation
problems on rawhide.
Josh Stone [Mon, 6 Jul 2015 18:35:51 +0000 (11:35 -0700)]
PR18555: prefer linkage_name to match the symtab
DW_AT_name is usually only the same as the symbol table for C. C++
names are mangled, which may be given by DW_AT_linkage_name. So if we
want to compare a DWARF subprogram to the symbol table by name, we
should prefer the linkage name when it's available.
This mattered especially for ppc64le, where query_dwarf_func was trying
to apply the global/local symbol offset. When we took a DWARF C++
function and tried to find that name in the symbol table for its offset,
there was no match, so the function wouldn't be resolved at all.
Now that lookup uses the linkage name. If there's still no match, like
with a stripped symbol table, then it falls through to just use DWARF's
entrypc as usual.
This patch also maintains the raw "addr" and offset "entrypc" separately
for symbol table functions, so for instance update_symtab can still
compare the original address.
David Smith [Thu, 2 Jul 2015 19:18:52 +0000 (14:18 -0500)]
Simplify a few syscall probes and handle RHEL5 kernels a bit better.
* tapset/linux/nd_syscalls.stp (nd_syscall.truncate): Simplified with fix
to long_arg() properly returning negative values.
(nd_syscall.keyctl): Ditto.
* tapset/linux/nd_syscalls2.stp (nd_syscall.quotactl): Only skip calls if
we're on a kernel >= 2.623, when sys32_quotactl() was introduced.
(syscall.truncate): Simplified.
* tapset/linux/syscalls2.stp (syscall.quotactl): Only skip calls if we're
on a kernel >= 2.623, when sys32_quotactl() was introduced.
(syscall.truncate): Matches more closely with the nd_syscall version,
which fixed a problem on RHEL5.
* tapset/linux/x86_64/syscalls.stp (syscall.get_thread_area): Added
support for RHEL5 kernels.
(syscall.set_thread_area): Ditto.
* tapset/linux/x86_64/nd_syscalls.stp: Ditto.
* testsuite/systemtap.syscall/fallocate.c: Properly handle ppc calls.
* testsuite/systemtap.syscall/keyctl.c: Handle keyctl() being
non-implemented.
* testsuite/systemtap.syscall/readahead.c: Handle ppc problems and glibc <
2.8 problems.
* testsuite/systemtap.syscall/readdir.c: Handle readdir() being
non-implemented.
* testsuite/systemtap.syscall/rt_signal.c (main): Handle
rt_tgsigqueueinfo() being non-implemented.
* testsuite/systemtap.syscall/uselib.c: Make sure we're on kernel >=
2.6.22, since before that uselib() could call quiet_ni_syscall(), which
we can't probe since it is in assembly.
David Smith [Wed, 1 Jul 2015 17:27:21 +0000 (12:27 -0500)]
Fixed PR18597 by returning negative values correctly from long_arg().
* tapset/powerpc/registers.stp (_stp_arg2): New function that fixes the
problem with _stp_arg() where long_arg() doesn't handle negative values
correctly. Callers can also force a 64-bit returned value.
(longlong_arg): Force a 64-bit return value.
* tapset/s390/registers.stp: Ditto.
* tapset/x86_64/registers.stp: Ditto.
* tapset/linux/nd_syscalls2.stp: Simplified nd_syscall.sync_file_range.
* tapset/linux/nd_syscalls.stp: Simplified nd_syscall.io_submit.
* testsuite/systemtap.syscall/pread.c: Add more 64-bit values to test.
* testsuite/systemtap.syscall/pwrite.c: Ditto.
Mark Wielaard [Tue, 30 Jun 2015 22:34:32 +0000 (00:34 +0200)]
Various fixes for test failures caused by implicit warnings.
Add various function and variable declarations and/or includes for
failures caused by new gcc warnings -Wimplicit-function-declaration
or -Wimplicit-int.
Mark Wielaard [Tue, 30 Jun 2015 19:54:28 +0000 (21:54 +0200)]
unwind.c (compute_expr): Don't fallthrough after div/mod/shr.
When processing DW_OP_div, DW_OP_mod or DW_OP_shr compute_expr
would accidentially fallthrough to the next case statement causing
the DWARF value stack to contain wrong values.
David Smith [Tue, 30 Jun 2015 20:24:54 +0000 (15:24 -0500)]
Two small stat code fixes found by source analysis.
* runtime/stat-common.c (_stp_stat_print_histogram_buf): Fixed small
potential overflow problem by widening values.
* runtime/stat.c (_stp_stat_init): Fixed missing 'va_end' call in an error
situation.
Martin Cermak [Fri, 26 Jun 2015 08:22:37 +0000 (10:22 +0200)]
PR18571: tapset support bpf and seccomp syscalls
NEWS: Deprecate seccomp's 'uargs' in favor of 'uargs_uaddr'.
tapset/linux/aux_syscalls.stp: New _seccomp_op_str() and _bpf_cmd_str().
tapset/linux/nd_syscalls.stp: New probe nd_syscall.bpf.
tapset/linux/nd_syscalls2.stp: Update probe nd_syscall.seccomp.
tapset/linux/syscalls.stp: New probe syscall.bpf.
tapset/linux/syscalls2.stp: Update probe syscall.seccomp.
testsuite/buildok/aux_syscalls-embedded.stp: Test _seccomp_op_str(), _bpf_cmd_str().
testsuite/buildok/nd_syscalls-detailed.stp: Test probe nd_syscall.bpf.
testsuite/buildok/nd_syscalls2-detailed.stp: Update test for probe nd_syscall.seccomp.
testsuite/buildok/syscalls-detailed.stp: Test probe syscall.bpf.
testsuite/buildok/syscalls2-detailed.stp: Update test for probe syscall.seccomp.
testsuite/systemtap.syscall/bpf.c: New testcase.
testsuite/systemtap.syscall/seccomp.c: New testcase.
David Smith [Thu, 25 Jun 2015 14:46:39 +0000 (09:46 -0500)]
Fixed PR18577 by making 'stap -l **' faster.
* tapsets.cxx (suggest_marks): Don't generate the suggestion list if we're
in either of the listing modes (-l/-L) and the verbose level is less
than 2.
(suggest_plt_functions): Ditto.
(suggest_dwarf_functions): Ditto.
* elaborate.cxx (suggest_functors): Ditto.
* elaborate.h: Updated suggest_functors declaration.
* tapset/linux/rpc.stp: Make some optional probes more optional.
* tapset/linux/scsi.stp: Ditto.
* tapset/linux/signal.stp: Ditto.
* tapset/linux/syscalls2.stp: Ditto.
* tapset/linux/nd_syscalls2.stp: Ditto.