Jonathan Lebon [Wed, 30 Oct 2013 15:12:45 +0000 (11:12 -0400)]
PR16019: treat errors from tapsets/functions as non-critical
Since we now include parsing errors in seen_errors, we get an increase
in num_errors() which we didn't have before. This would cause the
function passes_0_4() to stop at pass 1 even when parsing errors only
occurred in tapset/macro files (previously, we simply issued a warning
and kept going).
We now pass a new flag, errs_as_warnings, to the parsing methods so that
whenever any parsing errors occur while parsing tapset or macro files,
systemtap_session::print_error() will not count the error towards the
num_errors() value.
The net effect is that prior to pass 1b (user script parsing), any error
is printed (along with a warning) but not counted. Warnings are still
emitted, so that -W will still stop everything.
Thus, from the user's standpoint, there is no change in behaviour prior
to these series of patches.
Jonathan Lebon [Mon, 21 Oct 2013 13:35:48 +0000 (09:35 -0400)]
PR16019: improve duplicate error elimination
Previously, we relied on exact string matching when considering whether
to print an error or not. This had the disadvantage of not filtering out
whenever the same error occurred on many different lines. Also, error
chains, which often provided context, were cut off, causing errors to be
more cryptic.
With this series of patches, this is how behaviour is modified:
1. Errors are filtered out based on their source, i.e. the function
and line at which the error was thrown. This means that slightly
different error messages are still considered equivalent and
eligible for duplicate elimination.
2. Errors that get passed the filter are always printed with thei
full chain to provide maximum context.
3. When in verbose mode (just one -v), filtering is completely turned
off and all errors are printed out.
This patch simply installs the mechanisms through which these
modifications will take place in the next patches.
- main.cxx: print out number of suppressed errors on cleanup
- session.cxx: rework the print_error() functions to account for
chaining
- session.h: new function and variable declarations and modify
parse_error object to hold errsrc
- staptree.h: modify semantic_error object to hold errsrc
- util.h: add helper macros for throwing errors which include source
You can use the SEMANTIC_ERROR and PARSE_ERROR macros to ease error
throwing. These macros automatically pass the the function and line
number to the error constructor.
William Cohen [Tue, 29 Oct 2013 19:00:58 +0000 (15:00 -0400)]
Make the ARM _stp_store_deref and _stp_deref more like kernel ARM uaccess.h macros
The ARM uaccess.h macros for put_user and get_user do an explicit
typecast of the address into an unsigned long. This typecast is
important for the __stp_get_user_asm_half and __stp_put_user_asm_half
macros. Without it these macros attempt to do pointer and get an
incorrect address for the second byte of a 16-bit value.
William Cohen [Tue, 29 Oct 2013 16:44:49 +0000 (12:44 -0400)]
Make the 32-bit ARM get_user() and put_user() work for 16-bit quantities
The 32-bit ARM does not have instructions to perform 16-bit loads or
stores. The __get_user_asm_half and __put_user_asm_half macros
sythesize those operations. However, in most cases the pointers
passed into these macros are pointers to 16-bit types and the pointer
arithmetic will end up pointing at the next 16-bit quantity rather
than the second half (byte) of the 16-bit quantity. The macros need
to explicitly typecast the pointers as pointers to 8-bit quantities to
make the pointer arithmetic work out properly.
Josh Stone [Mon, 28 Oct 2013 22:42:46 +0000 (15:42 -0700)]
Add an example for tracking shared futexes
The existing futexes.stp does show shared futexes, but only by the
$uaddr which is specific to any given process. This futexes2.stp tries
to show shared futexes by the inode path+offset of the mapping. I found
this useful in examining stapdyn's pthread_mutexes in shm.
This also adds an inode_path() tapset function which tries to locate the
vfsmount containing an inode in the current namespace, and if found uses
task_dentry_path() to get a full path.
David Smith [Mon, 28 Oct 2013 21:35:43 +0000 (16:35 -0500)]
Fix 'syscall.accept' probe alias.
* tapset/linux/syscalls.stp (syscall.accept): Make sure optional probe
points are marked as optional at the top, since optionalness passes down
along alias definitions, not up.
David Smith [Mon, 28 Oct 2013 21:33:05 +0000 (16:33 -0500)]
Refix PR16058 by updating the old syscall.mmap2/nd_syscall.mmap2 probes.
* tapset/linux/i386/syscalls.stp: Make sure optional probe points are
marked as optional at the top, since optionalness passes down along
alias definitions, not up.
* tapset/linux/i386/nd_syscalls.stp: Ditto.
* tapset/linux/s390/nd_syscalls.stp:
* tapset/linux/s390/syscalls.stp: Ditto.
* tapset/linux/x86_64/nd_syscalls.stp: Ditto.
* tapset/linux/x86_64/syscalls.stp: Ditto.
Josh Stone [Fri, 25 Oct 2013 00:02:53 +0000 (17:02 -0700)]
Allow "read-mostly" stats to elide locks
In c_unparser::emit_lock_decls, we invert read/write locks for stats
types, since they only need a shared lock for writing updates but an
exclusive lock for aggregation to read. However, we didn't invert our
view of vcv_needs_global_locks across all probes, so it was impossible
for stats to ever get counted as "read-mostly" and elide locks.
Now the sense of vcv_needs_global_locks.read/written is swapped for
stats too, so those that are only aggregated in lock-free probes
(begin/end) can be completely lock-free.
Josh Stone [Thu, 24 Oct 2013 23:47:17 +0000 (16:47 -0700)]
stapdyn: Print dbug/warn/error messages with libc vsnprintf
The kernel runtime uses vscnprintf for these, which roughly follows
C99 rules, but the dyninst runtime was using _stp_vsnprintf, which is
not quite the same. Since some runtime code is shared, we need
compatible format strings, so just use libc's vsnprintf here.
Also, one shm_dbug was using %zi for off_t, which isn't always correct
since we #define _FILE_OFFSET_BITS 64, so use PRIi64 instead.
Josh Stone [Thu, 24 Oct 2013 23:00:38 +0000 (16:00 -0700)]
stapdyn: send context-free DBUG messages to stderr
Some debug messages occur outside of probe context, like shm_dbug,
so they would get an error trying to allocate the print buffer. In
the particular case of shm initialization, the transport memory
doesn't even exist yet!
Now if there's not an active context, DBUG just goes to stderr.
Josh Stone [Thu, 24 Oct 2013 22:44:19 +0000 (15:44 -0700)]
Filter duplicates out of globbed process names
For example, process("/lib64/libpthread*") matches libpthread-2.17.so
and libpthread.so.0, but the latter is just a symlink to the former. We
already canonicalized those names, but still had double the probes.
* tapsets.cxx (dwarf_builder::build): Track canonicalized name in a set
so we can skip globbed names that resolve to the same thing.
Josh Stone [Thu, 24 Oct 2013 21:56:06 +0000 (14:56 -0700)]
Fix canonicalize_file_name() leaks
This function allocates memory, but in a few places we weren't freeing
it. We have a resolve_path() to do this correctly for std::string-happy
code, and this patch adds free() for the rest.
Jonathan Lebon [Thu, 24 Oct 2013 14:02:25 +0000 (10:02 -0400)]
man: use SAMPLE macro in remaining docs
To be more consistent with the other man pages, all the remaining pages
had their SAMPLE macro modified in the same way (see commit 895c959).
- dtrace.1: add SAMPLE macro and replace manual RS/RE with it
- error::pass1.7stap: fix description on same line as .SH
- error::process-tracking.7stap: use new macro
- stapex.3stap: ditto
- stapprobes.3stap: ditto
Josh Stone [Wed, 23 Oct 2013 17:01:33 +0000 (10:01 -0700)]
Rename the 'examples' symlink to 'EXAMPLES'
This avoids a conflict with the way systemtap.spec is munging the
examples for %doc (rhbz 1022483), and in caps it will also be closer to
the top of the naive sorting in gitweb.
David Smith [Fri, 18 Oct 2013 18:49:45 +0000 (13:49 -0500)]
Fix PR16058 by updating the old syscall.mmap2/nd_syscall.mmap2 probe aliases.
* tapset/linux/i386/syscalls.stp: Break the 'mmap2' probe alias down into
subprobes, so that 'pgoff' can be converted to a number of bytes when
necessary.
* tapset/linux/i386/nd_syscalls.stp: Ditto.
* tapset/linux/x86_64/syscalls.stp: Ditto.
* tapset/linux/x86_64/nd_syscalls.stp: Ditto.
David Smith [Thu, 17 Oct 2013 21:23:58 +0000 (16:23 -0500)]
Fix PR15961 by avoiding use of symname(addr()) in syscall/nd_syscall tapsets.
* tapset/linux/nd_syscalls2.stp (nd_syscall.compat_rt_sigprocmask): Switch
from using 'symname(addr())' to 'ppfunc().
(nd_syscall.uname): Ditto.
(nd_syscall.rt_sigtimedwait): Break down into sub-probes to avoid
'symname(addr())'.
(nd_syscall.truncate): Ditto.
* tapset/linux/s390/nd_syscalls.stp (nd_syscall.mmap): Break down into
sub-probes to avoid 'symname(addr())'.
(nd_syscall.mmap2): Ditto.
* tapset/linux/s390/syscalls.stp (syscall.mmap): Break down into
sub-probes to avoid 'symname(addr())'.
(syscall.mmap2): Ditto.
David Smith [Thu, 17 Oct 2013 16:23:32 +0000 (11:23 -0500)]
Fix PR16058 by updating the syscall.mmap2/nd_syscall.mmap2 probe aliases.
* tapset/linux/syscalls.stp: Convert '$pgoff', which is a number of pages,
into a number of bytes.
* tapset/linux/nd_syscalls.stp: Ditto.
* testsuite/systemtap.syscall/mmap.c: Add testing for a non-zero offset
argument to mmap().
Jonathan Lebon [Thu, 17 Oct 2013 15:37:01 +0000 (11:37 -0400)]
man/stap.1: improve .SAMPLE macro
The .RE macro, used in the .ESAMPLE macro, restores the indentation
level to the default, not to what it was before the preceding .RS macro.
This meant that in already-indented regions (e.g. the .TP section of
--suppress-handler-errors or MAXMAPENTRIES), the text following .ESAMPLE
would be unindented one level too much.
The .SAMPLE macro now saves the current indentation level in a register
and the .ESAMPLE macro restores it. The macros were also modified so
that samples now have an empty line above and below to improve
legibility.
Old uses of .RS/.RE for samples were replaced by .SAMPLE/.ESAMPLE and
the printf examples section was unindented since the macro already
indents it once.
- man/stap.1: improve .SAMPLE macro, clean up usage
- man/stapvirt.1: improve .SAMPLE macro, use .PP instead of blank space
- man/stap-merge.1: improve .SAMPLE macro
- man/stap-prep.1: ditto
- man/stap-report.1: ditto
Josh Stone [Thu, 17 Oct 2013 00:15:31 +0000 (17:15 -0700)]
PR16057: Fix task_dentry_path crossing mount points
There were a couple ways that task_dentry_path failed on kernels which
have struct mount, i.e. >= kernel-3.3.
* tapset/linux/dentry.stp (real_mount): The (vfsmnt > offset) test fails
for most (all?) kernel pointers, as having the MSB set looks negative
in signed long comparisons. Explicitly allow "negative" pointers now.
(task_dentry_path): The @cast(mnt, "mount")->mnt_parent got us to
another struct mount, but we wanted the struct vfsmount. This is
available one step further in ->mnt_parent->mnt.
Jonathan Lebon [Tue, 3 Sep 2013 16:29:47 +0000 (12:29 -0400)]
add guest support files for libvirt/unix schemes
For systemd systems, the following files are used:
- stapsh@.service.in: template file for a service which automatically
respawns stapsh on a given virtio-serial port
- 99-stapsh.rules: udev rules file that instantiates (or terminates)
the stapsh service upon plugging in/out of virtio-serial ports
For SYSV systems, the following files are used:
- stapsh-daemon.in: libexec script file that automatically respawns
stapsh on a given virtio-serial port
- stapshd.in: init script to control stapsh-daemon instances
- 99-stapsh-init.rules: udev rules file that forces a configuration
reload upon plugging in/out of virtio-serial ports
- virtio_console.modules: ensures that the virtio_console module is
loaded at boot time if virtio-serial ports are detected. This is a
workaround for RHEL5, in which the module is not auto-loaded, but
does no harm in other systems such as RHEL6.
Jonathan Lebon [Fri, 16 Aug 2013 19:25:07 +0000 (15:25 -0400)]
stapvirt: initial release
The stapvirt binary is a tool for both users and stap. Users use it to
manage systemtap ports on their libvirt-managed domains. It allows them
to add, list, and remove ports. stap uses it to connect to a
libvirt-managed domain with a free systemtap port.
The stapvirt binary is only compiled if the libvirt-devel >= 1.0.2 and
the libxml-2.0 packages are installed.
- Makefile.am: build stapvirt if we HAVE_LIBVIRT and HAVE_LIBXML2
- configure.ac: PKG_CHECK_MODULES for libvirt and libxml2 and define
HAVE_LIBVIRT and HAVE_LIBXML2 conditionals
- man/Makefile.am: include stapvirt man page if we HAVE_LIBVIRT and
HAVE_LIBXML2
- man/stapvirt.1: created stapvirt man page
- stapvirt.c: created stapvirt source
Jonathan Lebon [Wed, 28 Aug 2013 16:13:50 +0000 (12:13 -0400)]
stapsh: use SIGIO to monitor port status
If the kernel supports it, a handler for SIGIO is installed. Upon any
event (e.g. host connection up/down), the handler updates a global
variable. This variable is used by host_connected() if possible, rather
than calling poll() everytime.
Jonathan Lebon [Mon, 26 Aug 2013 16:20:24 +0000 (12:20 -0400)]
stapsh: tweak error msgs
Make error messages more consistent with the rest of SystemTap (i.e.
capitalize the first word). Also, make die() respect host_connected()
and print out strerror().
Jonathan Lebon [Mon, 26 Aug 2013 16:01:19 +0000 (12:01 -0400)]
remote.cxx: use data option for unix
This commit actually enables the use of the new "data" option added in
commit 145fb2f. This is done by using a mini state machine. For now,
only the unix scheme uses this option. However, the upcoming libvirt
scheme will also make use of it.
The prefixing logic has also been revised. Now, cout is always flushed
so that e.g. printf("hello1\nhello2") will not only print out "hello1"
until ^C is done on stap. (This was most likely due to line-buffering).
Additionally, prefixing is now only done when necessary rather than
everytime (e.g. 'timer.s(1) { printf("hello") }' would add a prefix
before every 'hello').
Finally, prefixing now also accounts for multiple remotes, i.e. a new
line is automatically added and prefixed if data from a different remote
from the last was received.
- remote.cxx: add support for data commands and improve prefixing
- util.cxx: new split_lines() function
- util.h: split_lines() defn and new vector_has function
Jonathan Lebon [Mon, 26 Aug 2013 15:50:09 +0000 (11:50 -0400)]
stapsh: add data option
The data option causes stapsh to first send a "data" line before every
output from staprun. This has the key benefit of allowing us to
distinguish between stdout and stderr for schemes where the two are
mixed, such as unix and libvirt (upcoming in a future commit).
Jonathan Lebon [Mon, 26 Aug 2013 15:39:21 +0000 (11:39 -0400)]
stapsh: add option command
The new option command allows stap to change the behaviour of stapsh
depending on the scheme used. For now, only "verbose" is a valid option.
This is especially useful for the unix scheme, where we connect to an
already running instance of stapsh. Options are sent to stapsh right
after a successful handshake.
- remote.cxx: send options after successful handshake
- staprun/stapsh.c: add option command and verbose option
Jonathan Lebon [Fri, 16 Aug 2013 17:58:54 +0000 (13:58 -0400)]
stapsh-unix.exp: give time for socat to get set up
I've observed a few failures of this test in which stap gives the "No
such file or directory" error on connect() even though socat was started
properly. This happens the first time the test is run, but not
subsequent times. My theory is that socat takes some time to be loaded
and set up on the first run.
Looking at past testsuite results seem to indicate that this test failed
frequently, so this may have been a long-standing issue. Either way,
this small wait seems to do the trick.
Jonathan Lebon [Tue, 30 Jul 2013 14:08:48 +0000 (10:08 -0400)]
man/stap.1: small correction
This correction might be useful for those who want to parse the output.
Also add a small comment in stapsh.c to clarify the STAPSH_MAX_ARGS
comparison.
Frank Ch. Eigler [Sat, 12 Oct 2013 20:16:34 +0000 (16:16 -0400)]
PR9822: reduce timer load
Until this very moment, runtime/time.c's tsc-polling timer (the one
related to __stp_time_timer_callback) was being invoked every jiffy,
ie. something like 1000Hz, just to tune our per-cpu estimates of
tsc/real-time. This is excessive. Instead, use a much longer
interval (10Hz), and an even larger yet if cpufreq-change-notification
is in effect (0.1Hz).
* runtime/time.c (STP_TIME_SYNC_INTERVAL): New overrideable macro.
(__stp_init_time, __stp_time_timer_callback): Use it.
(__stp_time_cpufreq_callback): Refresh base_ns/base_cycles.
Josh Stone [Sat, 5 Oct 2013 00:12:04 +0000 (17:12 -0700)]
Fix minor errors found by clang analyzer
* dwflpp.cxx (dwflpp::iterate_over_plt): A bad plt_shdr pointer could be
dereferenced later in the function.
* mdfour.c (m): This static global pointer was retaining stack memory.
It's not needed at all - just pass an additional parameter as needed.
* stapregex-tree.cxx (range_union): In a twisted situation where old_b
is NULL, and old_a is valid but empty, then old_b was dereferenced.
This logic now stops trying to shortcut the empty case. Also, the
segment merging now uses std::merge, instead of doing it manually.
* staprun/stap_merge.c (main): Initialize num[];
* tapset-been.cxx (be_builder::build): Simplify 'priority' init;
* tapsets.cxx (dwarf_builder::build): Assert dw is always set.
* util.cxx (stap_spawn): Don't leave an unused 'cmd' local when SDT is
not enabled.
David Smith [Mon, 30 Sep 2013 15:12:47 +0000 (10:12 -0500)]
Fix PR15982 by making 'process.end' probes work reliably.
* runtime/linux/task_finder.c: Swith from using UTRACE_DEATH events to
using UTRACE_EXIT events, to avoid missing 'process.end' probes.
* tapset-utrace.cxx (utrace_derived_probe_group::emit_linux_probe_decl):
For CONFIG_UTRACE, use UTRACE_EXIT events instead of UTRACE_DEATH
events.
* testsuite/systemtap.base/proc_exec.exp: Tweak expected output.
Vincent Bernat [Sat, 28 Sep 2013 16:50:26 +0000 (18:50 +0200)]
PR15123: when PR15123_ASSUME_MFENTRY is set, don't check for -mfentry flag
-mfentry flag is recorded in `DW_AT_producer` only if
`CFLAGS=-grecord-gcc-switches` was used at compilation-time. We
provide PR15123_ASSUME_MFENTRY as an environment variable to override
this detection. The user is expected to set this variable only if it
is confident that that CFLAGS=-mfentry was used.
William Cohen [Fri, 27 Sep 2013 18:49:44 +0000 (14:49 -0400)]
Basic SystemTap support for aarch64
The SystemTap runtime libraries needs a number of machine specific
details for each architecture to be specified. This is an attempt to
provide the minimal set of them so that a "hello, world" SystemTap
script will compile.
Josh Stone [Thu, 26 Sep 2013 21:35:00 +0000 (14:35 -0700)]
testsuite: New sdt_casm for STAP_PROBE_ASM
An attempted fix for PR13974 had broken STAP_PROBE_ASM, because in that
case there's some ambiguity whether %note and %progbits should be
%%-escaped. We can't determine that in general, because it depends on
whether the user's asm() has operands. That fix was thus reverted, and
we're testing to make sure STAP_PROBE_ASM doesn't regress again.
We broke glibc's build on rawhide due to the way they use SDT_PROBE_ASM.
There's really no way to correctly use %note in that scenario, where we
can't know if there's operand parsing or not.
William Cohen [Thu, 1 Jan 1970 00:19:29 +0000 (00:00 -0019)]
Add am64 support
The "uname -m" reports arm64 as aarch64. Some translation is needed to
convert the name to the proper arch name so the appropriate Makefile
can be found. Also need to report the arch as a 64-bit architecture.
David Smith [Tue, 24 Sep 2013 16:20:22 +0000 (11:20 -0500)]
Define vzalloc() and vzalloc_node() for systems without them.
* buildrun.cxx (compile_pass): Add 'exportconf' tests for vzalloc() and
vzalloc_node().
* runtime/linux/alloc.c: Use STAPCONF_VZALLOC and STAPCONF_VZALLOC_NODE to
know if we need private versions of vzalloc() and vzalloc_node().
Josh Stone [Fri, 20 Sep 2013 23:57:27 +0000 (16:57 -0700)]
Fixes for CC=clang CXX=clang++
We can compile almost everything with clang now, except:
- loc2c.c due to nested functions.
- stapdyn/ because dyninst pulls in __gnu_cxx::hash_map for GNUC < 4.3,
which triggers -Wdeprecated in libstdc++-4.8.1.
(clang #defines itself as GNUC 4.2.1.)
* loc2c-test.c (fail): Indicate that this is itself a formatted
function, so it doesn't complain about non-const 'fmt' passed to
vfprintf. (The callers are in fact passing const strings.)
* session.cxx (systemtap_session::parse_cmdline): Added braces so
that the if-for-if-else doesn't have a dangling else.
* stapregex-dfa.h: translator_output is a class.
* stapregex.h: translator_output is a class; dfa is a struct.
* tapset-method.cxx (java_builder): Remove unused cache_initialized.
Josh Stone [Fri, 20 Sep 2013 23:08:28 +0000 (16:08 -0700)]
testsuite: split std_asm out of std_misc.exp
The asm part of sdt_misc.exp was not really dependent on any other part
of that testcase, so we can let it stand alone for direct testing. Its
reported name in test results is left as "sdt_misc", so it can still be
easily compared to previous test runs.
Josh Stone [Fri, 20 Sep 2013 23:00:44 +0000 (16:00 -0700)]
SDT: Use %foo for section types rather than "foo"
GAS can take section types as "foo", @foo, or %foo. We had the quoted
form, but this is not documented, and llvm-mc doesn't support it. Using
@foo starts a comment on ARM, so we're left with %foo.
For note, we have to escape it to %%note for the asm()'s operand
expansion, but leave it as %note for pure assembly. For progbits, there
are no operands, so it's %progbits either way.
Thanks to Martin C. Martin for first reporting this to LLVM[1], and then
returning to systemtap with a suggested compatibility fix.
[1] http://llvm.org/bugs/show_bug.cgi?id=17270
William Cohen [Thu, 19 Sep 2013 20:56:00 +0000 (16:56 -0400)]
Fix up testsuite/Makefile.am RUNTEST variable
The '"'s in the definition of RUNTEST in testsuite/Makefile.am was
making causing the test for DEJAGNU to fail. There didn't seem to be
a reason to keep the quotes around, so they were removed and "make
installcheck" works again.