Jonathan Lebon [Wed, 28 May 2014 15:12:18 +0000 (11:12 -0400)]
BZ1099757: vars.exp: fix and strengthen
Since PR16615 (merge commit 4636ca3), linenos in statement probes have
become more precise and stringent. As a result, vars.exp was failing
because it was trying to probe a statement at a lineno with no line
records.
We fix this by first retrieving the first valid lineno using stap -l.
The test has also been refactored in general to be more robust. We can
now more easily add more functions to test (e.g. we now test both
bio_copy_user() and vfs_read()).
Jonathan Lebon [Fri, 16 May 2014 14:45:27 +0000 (10:45 -0400)]
list.exp: fix regexes and strengthen
Fix regular expressions so that we expect the full path from stap rather
than just process("stap"). Also strengthen the testcase by making
matches against all output lines, rather than just checking if a single
line matches.
Jonathan Lebon [Tue, 13 May 2014 21:15:01 +0000 (17:15 -0400)]
derive_probes(): properly handle optionality
We previously overrode a location's optionality if it was derived from a
globby process probe. However, this causes further probe points to be
derived to also be optional, even though the 'higher level' probe point
had its optionality restored to what the user really wants. The end
result was that we would list probes as not optional even though the
endpoint probe was optional.
What we really mean when we say process probes born out of a higher
globby process probe should be optional is that only some of the matched
binaries will actually resolve, so it's OK if some don't resolve.
However, those that *do* resolve should respect the user's wish with
regards to optionality.
To do this, we must not override the probe_point's optional flag in
derive_probes.
This is mostly a cosmetic change, although some runtime elements do rely
on optionality to decide whether to emit a warning upon failing to
register a probe.
This patch also makes process.library probes born out of a higher globby
library probe follow the same logic.
The line at which 'mylabel' occurs is not actually associated with any
line records. Thus, when using the new iterate_over_srcfile_lines(),
specifying the lineno of the label results in no records found, whereas
prior to PR16615, the next lineno would be picked up.
Regardless, the previous mechanism of using iterate_over_srcfile_lines()
to then call back to query_srcfile_label() which in turn called
iterate_over_labels() was a roundabout way of making sure that the
specified lineno lies in a filtered function's DIE.
We now rework iterate_over_labels() to directly handle lineno matching
when necessary. Supported cases:
process.function("foo@file.c:N").label("*")
--> matches label in foo at N
process.function("foo@file.c+N").label("*")
--> matches label in foo at N + decl_line of foo
process.function("foo@file.c:N-M").label("*")
--> matches labels in foo between N and M
process.function("foo@file.c:*").label("*")
--> matches all labels in foo
This patch also pushes down the priority of "func@file:N" type probes so
that if a .label or .callee[s] is present, then the relevant
iterate_over_*() is called, rather than iterate_over_srcfile_lines().
Jonathan Lebon [Thu, 15 May 2014 22:06:17 +0000 (18:06 -0400)]
alias probes: mark final aliases as well-formed
When building probes from aliases, we mark them as well-formed if they
reached their final derivation so that in listing mode, we stop at this
final alias form.
Jonathan Lebon [Wed, 7 May 2014 18:16:24 +0000 (14:16 -0400)]
library probes: also try non-NEEDED libraries
This patch adds back support for non-NEEDED libraries even when the user
supplied a non-canonical path. However, a WARNING is emitted to notify
the user. This WARNING can be suppressed by specifying the fully
resolved path to the library.
Jonathan Lebon [Mon, 26 May 2014 16:01:18 +0000 (12:01 -0400)]
process probes: throw error upon unresolved executable
When find_executable() fails to resolve something, it simply returns the
same path it was fed. Depending on the context, the caller may be able
to deal with an unresolved path.
In the case of process probes for dwarf_builder and other base_query
users, dealing with it meant simply that the dwflpp failed to find the
executable which meant iterate_over_modules() never even took off (and
we thus yielded the good ol' "no match" error).
However, in the case of library probes, this may not always work because
if the library path is fully specified, we will switch to it without
resorting to iterate_over_libraries() (see also previous patch) and
without ever actually checking if the process path was fully resolved,
thus allowing a probe such as the following to be created:
So instead, we add explicit checks to verify that process probes were
properly resolved by find_executable(). As an added benefit, users get a
clearer reason of why the probe point failed:
BEFORE:
semantic error: while resolving probe point: identifier 'process' at <input>:1:7
source: probe process("asdf").library("mylib.so").function("foo") {}
^
semantic error: no match
AFTER:
semantic error: while resolving probe point: identifier 'process' at <input>:1:7
source: probe process("asdf").library("mylib.so").function("foo") {}
^
One of two things happens to library probes the first time we enter
dwarf_builder::build():
- If the full path can be resolved using find_executable(), then we
would right away switch focus to the library, rather than the
process.
- Otherwise, we would focus on the process and then use
iterate_over_libraries() to find a library match.
These decisions are made in two spots: in dwarf_builder::build(), prior
to creating the dwflpp object, and in base_query::base_query().
When we resort to iterate_over_libraries(), new probes are created for
each matching library. And since the path inserted in those new probes
is the full path, find_executable() will find it upon recursing into
dwarf_builder::build().
One issue is that if find_executable() succeeds on the first time, we
will focus on the library, regardless of whether the library is NEEDED
or not. Whereas if find_executable() fails, only NEEDED libraries could
ever be picked up by iterate_over_libraries().
This is mostly fine, except that it creates an inconsistency. For
example, let's say executable 'bar' depends on 'libfoo.so' located in
/lib/. Doing a "libfoo.so" library probe would prioritize a libfoo.so
file in the current directory before the one in /lib/, which is the
'real' one. On the other hand, doing "libfoo.*" would match the
libfoo.so file in /lib/ because of the glob char.
It also fixes the potential issue of having a fully specified library
path with a bad process path: we would focus on the library yet in
query_module() try to iterate_over_libraries() on the library, thinking
that the library was not resolved.
In this series of patches, we change the behaviour as follow
- Only switch to the library on the first entry of
dwarf_builder::build() if the path is already fully resolved.
Otherwise, always use iterate_over_libraries().
- If iterate_over_libraries() fails, then try a direct
find_executable() invocation, and warn that the found library is
not a NEEDED one.
This patch does the first part only. The second part is done in the next
patch.
Jonathan Lebon [Tue, 6 May 2014 20:39:04 +0000 (16:39 -0400)]
library probes: don't add libraries to visited_modules
The dwarf_query's visited_modules set is used to keep track of which
modules are visited for suggestions. It works by emptying out the visited
modules into the dwarf_builder's modules_seen after we're done iterating
through the modules. By the end, modules_seen contains the full list of
modules to suggest from.
Since .library(glob) probes resolve to multiple probes for each matching
library, dwarf_builder's modules_seen will already pick each of them up
from the individually created dwarf_query objects.
So in the end, the modules_seen from which we make suggestions will
already contain them.
Almost all of the users of final_function_name() call it with a const
char* for the filename, rather than a string. There may be cases where
the filename is NULL (e.g. if the function was derived using the
symtab), in which case we don't want to cast to a string.
Jonathan Lebon [Wed, 14 May 2014 14:48:39 +0000 (10:48 -0400)]
query_plt_statement(): add well-formed probe point
If the user provides a .plt[(string)] probe, then we'll end up in
query_one_plt during plt iteration, in which we create a specific_loc
where we replace the possibly globby string with the found plt. Mark
this specific loc as well-formed.
If the user provides a .plt[(string)].statement(number) probe, then
we'll end up in query_plt_statement() directly, in which case we need to
create the final well-formed probe point in which we omit the .plt part.
In commits e772a6e and 9891c54, we improved the way query_callee() and
query_label() add intermediate probes to the derivation chain by playing
with the dwarf_query. This technique will be required many times in
other places that we will need to fix up.
Therefore, we create two new dwarf_query helper functions. The
mount_well_formed_probe_point() function simply sets the current
base_loc and base_probe of the dwarf_query object to a new one, which is
denoted well-formed.
At first, this new location is simply a copy of the previous location,
with the exception of a fully resolved process/module path. Then,
depending on the probe point type, specific functors are replaced by
their final/non-globby versions to create a truly well-formed probe
point.
More helper functions to edit the functors will be added in the next
patch.
The unmount_well_formed_probe_point() refocuses the dwarf_query to the
previous base_loc/base_probe so that further probe points can be
derived.
Jonathan Lebon [Wed, 14 May 2014 14:31:07 +0000 (10:31 -0400)]
sdt_query: set well-formed intermediate probe
The specific_loc intermediate probe created in
sdt_query::convert_location() is already well-formed. We mark it as such
so that listings will use this probe.
Jonathan Lebon [Tue, 13 May 2014 20:18:43 +0000 (16:18 -0400)]
simplify derived_probe::script_location()
Rather than relying on yucky heuristics, we go the explicit way. The
script_location() function now relies on the new probe_point member
'well_formed', which is true if the probe point represents the 'final'
script-level expression, after wildcard expansion.
Since only each individual probe builder knows what constitutes a
well-formed probe point, it will be up to those to set the 'well_formed'
flag as needed. If the probe point that we want to list is also the
final one, which is the case for many simpler probe points, then no
further work is needed. The following patches modify the appropriate
builders to do just this.
David Smith [Wed, 21 May 2014 19:09:41 +0000 (14:09 -0500)]
Fix BZ16960 by handling !CONFIG_USER_NS better.
* buildrun.cxx (compile_pass): Convert the from_kuid_munged() test to a
compile test, instead of an export test.
* runtime/linux/runtime.h: Restore original CONFIG_USER_NS support.
* runtime/linux/autoconf-from_kuid_munged.c: New autoconf test.
Frank Ch. Eigler [Wed, 21 May 2014 17:43:48 +0000 (13:43 -0400)]
linux runtime: switch to vmalloc for context[] element allocation
The new eventcount.stp changes can result in relatively large context
structures (33712 bytes each on f19/x86-64), which the stp_kmalloc
widget can sometimes fail to allocate. Switching to the numa-aware
vmalloc front-end lets these allocations succeed.
Frank Ch. Eigler [Sat, 17 May 2014 20:15:01 +0000 (16:15 -0400)]
linux/runtime.h: more CONFIG_USER_NS fallout
Partially unrolling commit 7d35a948401, because on modern kernels
(3.14+), the from_k?id_munged functions are defined as inlines, and
should be used regardless of CONFIG_USER_NS etc. The buildrun.cxx
exportedness test for from_kuid_munged would fail, triggering our
runtime to use a do-nothing macro instead.
Until a more programmatic way is found for those unidentified older
kernels that have CONFIG_USER_NS but no from_k?id_munged inlines,
for now a user will have to work around the build problem with
Jonathan Lebon [Mon, 12 May 2014 13:50:20 +0000 (09:50 -0400)]
sdt_varname.exp: small tweaks
1. Only run testcase if installtest
2. Uncomment the (mistakenly commented out) code that cleans up the
testing directory upon exit
3. Enable -fPIC when compiling library, which can yield different forms
of operands, thus improving the testing range
Jonathan Lebon [Fri, 9 May 2014 19:57:28 +0000 (15:57 -0400)]
Merge branch 'jlebon/pr13296'
This merge gives SystemTap the ability to decode SDT operands that refer
to symbols. There are also a few code cleanups, such as:
- Removal of SDT probing code that supports a hypothetical kernel
mode.
- Removal of old unused symbol table code for reading in symbols by
reading output from nm or from a file.
- Major refactoring of the sdt_uprobe_var_expanding_visitor::
visit_target_symbo_arg() function to make it easier to understand
the overall structure.
- Strengthening of sections/note entry checks before parsing probe
information.
Jonathan Lebon [Thu, 1 May 2014 15:59:15 +0000 (11:59 -0400)]
PR13296: allow SDT operands using VARNAME
We may encounter SDT operands of the form [OFF+]VARNAME[+OFF][(REG)]
which refers to the address of a object in the symtab. On x86_64, the
REG can be '%rip' to denote RIP-relative addressing.
To do this, we also pass the current dwflpp object to
sdt_uprobe_var_expanding_visitor, which then tries symbol resolution
using the symbol table if it meets upon such an SDT operand.
Jonathan Lebon [Thu, 1 May 2014 15:18:22 +0000 (11:18 -0400)]
symbol_table: remove read_* methods
These methods of retrieving symbols haven't been in use since 2008 (at
least according to the date of the comment mentioning its abandonment).
Rather than having it slowly rot away, let's remove it. Git will
remember it for us if we ever need it back.
Jonathan Lebon [Wed, 23 Apr 2014 15:45:12 +0000 (11:45 -0400)]
setup_note_probe_entry(): strengthen note checks
In dwflpp::iterate_over_notes(), we call back to
sdt_query::setup_note_probe_entry() for every note entry found,
regardless of section name or note name, which is fine.
However, on the callback side in setup_note_probe_entry(), we never
actually check that the section name or note name is correct; we only
check that the note entry type was what we expected.
This means that it could have been possible for setup_note_probe_entry()
to try to blindly parse data from a note entry which was not even a
stapsdt note, which could be dangerous.
This patch adds explicit checks that the section name and the note name
are what we expect.
Jonathan Lebon [Thu, 8 May 2014 19:16:36 +0000 (15:16 -0400)]
callee.exp: simplify shlib testing by using -rpath
We previously started stap separately from the target process because we
needed to set the LD_LIBRARY_PATH variable for it to run properly.
However, if we use the linker option -rpath to add the current directory
to the library path (as is done also in unprivileged_myproc.exp), this
is no longer necessary, thus greatly simplifying the testcase.
William Cohen [Tue, 6 May 2014 20:01:54 +0000 (16:01 -0400)]
Flip condition so proper default picked for STAP_NOP in testsuite/sys/sdt.h
A number of the uprobes code failed to build on aarch64 because the
define for STAP_NOP would default to the unusual "nop 0". The test
has been flipped around so the default is usual "nop" if there is
not something explict.
David Smith [Mon, 5 May 2014 21:25:25 +0000 (16:25 -0500)]
Fix more compile error when we don't have a dwarf unwinder.
* runtime/linux/runtime.h: Fix compilation on systems without
STP_USE_DWARF_UNWINDER defined.
* runtime/unwind/unwind.h: For systems without STP_USE_DWARF_UNWINDER
defined, just define a dummy unwind_context structure along with the
real unwind_cache structure.
David Smith [Mon, 5 May 2014 20:46:40 +0000 (15:46 -0500)]
Fix compile error when we don't have a dwarf unwinder in _stp_tack_kernel_print
* runtime/stack.c (_stp_stack_kernel_print): Use 'c->kregs' instead of
nonexistent 'regs' variable. This code is only compiled when
STP_USE_DWARF_UNWINDER isn't defined.
beginnner's guide configury: compatibility with older publican
Older publican doesn't support the --pdftool=fop option. It's too
much hassle to autoautoconf this (considering the existing fop-related
conditionals at the top level configure.ac), so instead here we try
publican with and then without the --pdftool=fop option.
Stan Cox [Mon, 5 May 2014 14:24:20 +0000 (10:24 -0400)]
Add pyparsing to dtrace
* dtrace.in: pylint cleanups.
(_HeaderCreator): Moved from _provider.
(_PypProvider): New. The pyparsing parser.
(_ReProvider): Moved from _provider. The regexp parser.
(main): Add pyparsing awareness.
Jonathan Lebon [Fri, 2 May 2014 14:19:06 +0000 (10:19 -0400)]
PR16894: new wordexp.exp to check -c COMMAND handling
This simple testcase checks that invalid -c COMMANDs for unspecified
process probes are properly handled (e.g. prior to commit eb94071, we
segfaulted).
Jonathan Lebon [Fri, 2 May 2014 13:43:07 +0000 (09:43 -0400)]
PR16894: give better errors re. -c commands
This patch allows SystemTap to make the distinction when reporting an
error between no -c command given, and a given but invalid -c command.
- man/stapprobes.3stap: introduce 'unspecified' verbiage and clear up
restrictions on unspecified process probes
- session.cxx: cmd_file() now throws errors upon wordexp() failures
- tapsets.cxx: (1) handle possibility of error from cmd_file() and chain
with own error if it occurs
(2) separately handle no -c command / empty file
(3) introduce 'unspecified' verbiage in error messages
and point to [man stapprobes]
- tapset-perfmon.cxx: ditto