Frank Ch. Eigler [Wed, 26 Jan 2022 19:10:38 +0000 (14:10 -0500)]
PR28804: tune default stap -s ## buffer size on small RAM machines
Insert a forgotten division by num_online_cpu() to adjust downward the
calculated bufsize. Tweak normal defaults back to 128 * 2 * 64K
(16MB) per CPU, as the stap man page indicates. This may need further
tweaking when balancing against staprun consumption performance, but
at least we have the docs lined up with the code at the moment.
Serhei Makarov [Fri, 21 Jan 2022 23:21:46 +0000 (18:21 -0500)]
gcc12 c++ compatibility re-tweak for rhel6: use function pointer instead of lambdas instead of ptr_fun<>
Saving 2 lines in ltrim/rtrim is probably not a good reason to drop
compatibility with the RHEL6 system compiler. Actually declaring a
named function and passing the function pointer is compatible with
everything.
William Cohen [Tue, 18 Jan 2022 03:02:44 +0000 (22:02 -0500)]
Graceful continuation when not enough memory available for liveness analysis
The dyninst parsing of binaries can take a significant amount of
memory. On machines without enough memory to parse a large binary we
want the analysis to fail gracefully with a warning that the liveness
analysis was unable to run and continue on rather than immediately
exiting with a std::bad_alloc exception.
William Cohen [Fri, 14 Jan 2022 19:00:02 +0000 (14:00 -0500)]
configure finds appropriate default 32-bit or 64-bit Dyninst libraries
Earlier versions of the systemtap configuration would just include
two -L paths to both 32-bit and 64-bit versions of the Dyninst
libraries. However, attempting to link a 32-bit library with a 64-bit
build (and vice versa) may cause the build to fail. This revision of
the configure tests determines which default Dyninst library works
with the compiler being used and selects it.
The configure can't bindly use ${libdir}/dyninst to select the path to
the default Dyninst libary as this only selects the ${prefix}/lib64 on
appropriate machines if the prefix is /usr. If the prefix is set to
something else, ${libdir} is always ${prefix}/lib. The would cause
the build to attempt to link with nonexistent Dyninst libaries in the
${prefix} directory. If systemtap needs to be use a version of
Dysninst in a non-standard place, the --with-dyninst=<path_to_dyninst>
should be used.
Stan Cox [Fri, 7 Jan 2022 17:01:49 +0000 (12:01 -0500)]
Standardize dyninst include file use.
Change stapdyn to use the more standard '#include <dyninst/*.h>' form
and the standard include path. Adjust configure so that
--with-dyninst=PATH works, assuming PATH is a /usr style path laid out
in standard linux form.
It appears that various versions of gcc continue to show signs of
confusion at our newly offered asm-operand alternatives for floating
point sdt.h marker parameters.
Stan Cox [Wed, 1 Dec 2021 21:19:22 +0000 (16:19 -0500)]
Handle user supplied sdt probe argument template
User supplied templates were erroneously removed by commit eaa15b047,
which complicated the template expansion. To do the above the
expansion of STAP_PROBE_ASM(provider, fooprobe,
STAP_PROBE_ASM_TEMPLATE(3)) adds an unused argument:
STAP_PROBE_ASM(provider, fooprobe, /*template expansion*/ "%[SDT..]..",
"use _SDT_ASM_TEMPLATE") A supplied template
STAP_PROBE_ASM(provider, fooprobe, "4@%rdx 8@%rax") is left alone. If
the varargs has 2 args (the fake "use ..") then macro expansion
inserts the expanded string, otherwise "4@.." becomes an ascii op.
Frank Ch. Eigler [Sat, 20 Nov 2021 03:22:45 +0000 (22:22 -0500)]
configury: let python3 be python3
Our baroque heuristics for identifying python2/3 under their various
historical aliases is showing its age. On some modern distros,
/usr/bin/python is to be positively NOT used. Fixing configure.ac
$PYTHON3 search to only look for python3, and not even consider
$PYTHON_UNKNOWN. At some point we'll want to simplify further, and
get rid of python2 remnants.
Frank Ch. Eigler [Mon, 15 Nov 2021 02:59:05 +0000 (21:59 -0500)]
PR28449: runtime/transport: bump up _stp_subbuf_size to 64K
After commit cd48874296, it was reported that the kernel-side I/O
buffers could be exhausted more easily by the some tests in the
testsuite. This resulted in "bufhdr corrupted ..." type messages
coming from the syscall.exp test case for example.
We bump up the $subject variable, so without "stap -s XYZ", the
default kernel->user transport will consist of 256 subbufs of 64K
each, per cpu. (When a user does supply -s XYZ, the number gets
overridden, and the subbuf size may be quite a bit larger.) With the
new default, the syscall.exp suite runs bufhdr-clean on an 8cpu box.
Frank Ch. Eigler [Sun, 14 Nov 2021 21:28:12 +0000 (16:28 -0500)]
PR28557: module probe insertion on modern kernels
For reasons not completely understood, kernels in the 5.* range have
stopped taking module kprobes registrations of the form the runtime
has offered them. New code in runtime/linux/kprobes.c
(stapkp_prepare_kprobe) now tried to fully relocate addresses itself,
rather than letting the kernel try it with symbols / kallsyms.
Added more diagnostic prints in nearby symbol/kprobe code paths and a
few more error checks in transport.
Serguei Makarov [Thu, 4 Nov 2021 15:33:36 +0000 (11:33 -0400)]
procfs_bpf.exp PR28544: fix string handling error
This was causing a procfs_bpf.exp failure on recent RHELs
as a non-zero-padded string passed from the procfs pipe
caused buffer garbage to be passed to the stap script.
Serguei Makarov [Thu, 4 Nov 2021 12:53:11 +0000 (08:53 -0400)]
man/stap.1.in stapregex: clarify POSIX ERE features
Although stapregex was based on re2c, its current feature set implements
POSIX Extended Regular Expressions. However, a lot of POSIX ERE docs
that come up in a web search include extra stuff (e.g. the Boost doc).
Refer the user more clearly to egrep(1) for the exact list of features.
pass-2 elaborate: autocast: suppress "Potential type mismatch in reassignment"
Analysis indicates this warning is not that helpful, and extra
diagnostics just added for autocasting should help work around
polymorphism issues that this warning would have identified earlier.
William Cohen [Tue, 2 Nov 2021 15:20:27 +0000 (11:20 -0400)]
Add -faligned-new to CXXFLAGS when available to compile dyninst 10.1 code
Fedora 30 and 31 have Dyninst 10.1. The code added to dyninst
by commit a00056bec6a0265afb592944dde2ff461b525e8d makes some data
structures cache aligned (128 bytes). The c++ compiler ends up flagging that
as an error:
CXX stap-analysis.o
/notnfs/smakarov/stap-checkout/analysis.cxx: In constructor ‘analysis::analysis(std::string)’:
/notnfs/smakarov/stap-checkout/analysis.cxx:59:37: error: ‘new’ of type ‘Dyninst::ParseAPI::SymtabCodeSource’ with extended alignment 128 [-Werror=aligned-new=]
59 | sts = new SymtabCodeSource(name_str);
| ^
/notnfs/smakarov/stap-checkout/analysis.cxx:59:37: note: uses ‘void* operator new(std::size_t)’, which does not have an alignment parameter
/notnfs/smakarov/stap-checkout/analysis.cxx:59:37: note: use ‘-faligned-new’ to enable C++17 over-aligned new support
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:1724: stap-analysis.o] Error 1
To make the compiler happy adding a -faligned-new option to the
CXXFLAGS when it is available. This issue does not affect later
releases of dyninst as the problem code was removed from dyninst in
git commit d233ae7596cd74201d634c2f0f7d7a0e3d628e79.
Our type-detail inference (tagging some staptree nodes with DWARF DIE
pointers) in pass 2 was not very verbose about its work, making
type-mismatch warnings harder to diagnose. Add a ::print() virtual
function to exp_type_details() and derived classes. Call it from
resolve_details(), renamed from resolved_details().
Frank Ch. Eigler [Sat, 30 Oct 2021 23:50:21 +0000 (19:50 -0400)]
staprun/relay.c: bypass compiler warning
Newly added code did a read(2), deliberately ignoring how many bytes
were read. Some gcc versions complain. Phrase the same operation
differently to make gcc happier.
Frank Ch. Eigler [Sat, 30 Oct 2021 19:21:31 +0000 (15:21 -0400)]
PR28449: fix cross-cpu message ordering
Correct unintended loss of cross-cpu message total-ordering via commit 8819e2a04596d by packetizing normal printf() traffic. This also
restores bulk mode / stap-merge operation to same as before. staprun
now has a builtin merge equivalent built on threads, with some
message-loss fault-tolerance. A new "dumpalot" test produces ample
output with varying "-s BUFFER" runs. While in the vicinity, the old
"relay_old.c" (<=rhel5) file is nuked; it was not even compiled.
William Cohen [Thu, 28 Oct 2021 20:13:56 +0000 (16:13 -0400)]
Adjust analysis code to compile with older Dyninst 9
The Block getInsn method and InsnLoc constructor have different
signature in older Dyninst 9. To hide those differences between
version of Dyninst just put the method invocation in the constructor
and let the compiler figure out which type the parameter is.
William Cohen [Thu, 28 Oct 2021 17:38:02 +0000 (13:38 -0400)]
Tolerate dyninst10's need for -lboost_system in analysis.cxx
The analysis code uses dyninst. As a result the new analysis code in
the translator will need to link in the boost_system library like
commit 891810c246d6de05a2df80c5b3e9f9aaa13231f7 does for stapdyn.
William Cohen [Wed, 27 Oct 2021 17:49:12 +0000 (13:49 -0400)]
Make the liveness warning 32-bit agnostic
On 32-bit platforms Dwarf_Addr is long long unsigned int which doesn't
match the %lx (long unsigned int) print format to print the warning.
Cast the variable as (void *) so %p format can be used to print its
value on both 32-bit and 64-bit platforms.
William Cohen [Wed, 13 Oct 2021 19:56:21 +0000 (15:56 -0400)]
Assert that anything other than 32-bit or 64-bit processors will not be seen
One last diagnostic print to remove. In this case the mapping between
dwarf register and Dyninst register name needs to take into account
whether this is 32-bit or 64-bit code. However, there is a default in
the switch case to catch the anything other than 4 or 8 bytes. If the
code see something other than one of those two values, something is
very wrong. Figured best to just have an assert stop things, so the
problem is examined.
William Cohen [Wed, 13 Oct 2021 19:27:30 +0000 (15:27 -0400)]
Add caching for the liveness data structure
Dyninst already caches information to make future queries at the same
location less expensive for an executable. Thus, it is enough for the
code to cache the LivenessAnalyzer data on a per executable basis.
This information is stored in the cachedLivenessInfo member of
LivenessAnalyzer class.
William Cohen [Thu, 30 Sep 2021 19:18:59 +0000 (15:18 -0400)]
Use the elf_path in mod_info as that always points to the executable
For kernel probes q.dw.module_name just lists "kernel" rather than
the actual path to the binary. The mod_info elf_path has more accurate
path to the actual executable, using that instead.
William Cohen [Fri, 24 Sep 2021 16:04:21 +0000 (12:04 -0400)]
Eliminate unnecessary CodeRegion discovery
Documentation on findFuncs method seemed to require a CodeRegion passed in.
However, Dyninst developers said in this case can just pass in a NULL
(https://github.com/dyninst/dyninst/issues/1102). Simplified
the code to eliminate the uneeded CodeRegion information.
William Cohen [Wed, 22 Sep 2021 21:34:25 +0000 (17:34 -0400)]
Do liveness analysis after location information set and use regno info
Initially the call to the liveness analysis was being done before
the location information was setup. Relocated the call after the
location information has been setup.
Corrected liveness analysis code to use the register number. Also
added print out the function the liveness analysis is being run on.
Note that for probes on inlined functions this name will be different
than the probe point function name, it will be the actual function
that the code was inlined into.
William Cohen [Tue, 21 Sep 2021 19:36:08 +0000 (15:36 -0400)]
Add support to handle 32-bit x86 binaries
Dyninst liveness analysis treats the analysis of 32-bit x86 and 64-bit
x86_64 code differently. In dyninst the register names are different.
Thus, trying to get the liveness results for rdi (the 64-bit register
name) on 32-bit analysis will fail. The analysis also needs to treat
32-bit and 64-bit a bit differently as there are difference in the
calling ABIs.
Right now the code dummy up using edi/rdi register which is the holds
the first argument in the register. Doing this because the
location_context cts doesn't hold information outside the variable name.
William Cohen [Mon, 20 Sep 2021 13:46:04 +0000 (09:46 -0400)]
Show the liveness information at points being probed
This code is work-in-progress and to check whether the liveness
analysis is working. Currently, the location context information for
the variable isn't being passed in and the setup of that is disabled.
William Cohen [Mon, 30 Aug 2021 18:32:57 +0000 (14:32 -0400)]
Update configure machinery and add files to hold Dyninst analysis
The analysis uses tooling in Dyninst to do the analysis. However,
Dyninst is not available for some architectures. Needed to make the
configuration machinery only enable the Dyninst-based analysis for
supported machines such as x86-64 and aarch64.
The analysis.cxx is not currently connected to anything and doesn't do
any analysis. For the moment it uses a couple Dyninst functions
verify that Dyninst libraries are being linked in. This has been
tested to compile on both architectures that support Dyninst (x86_64)
and do not support Dyninst (32-bit Arm).
William Cohen [Mon, 25 Oct 2021 15:32:46 +0000 (11:32 -0400)]
Update syscall_any number<->name maps to include syscallent-common.h entries.
Some syscalls information had been moved to syscallent-common.h in
the strace code. Need to use the #include in the syscallent*.h files
to get those entries.
William Cohen [Mon, 25 Oct 2021 15:23:47 +0000 (11:23 -0400)]
Pull in the includes to get strace syscall info from syscallent-common.h
There are now a number of syscall in strace that are described in
src/linux/generic/syscall-common.h which is pulled into a number of
different architecture syscallent*.h includes. Don't want to miss
those additional syscalls in the mappings.
Serguei Makarov [Thu, 14 Oct 2021 20:44:23 +0000 (16:44 -0400)]
stap_run.exp: accept 'WARNING: Child process exited due to signal'
This was causing numerous spurious FAIL outcomes on RHEL[89].
The extra "WARNING: Child process exited due to signal" is not an error,
it's merely stap communicating to the user that the child
process was interrupted, which is exactly what we do.
Not sure why it's printed on some systems and not others.
XXX There may be a deeper issue in that the tiny do-nothing binary
probed by the testcase (e.g. testsuite/systemtap.base/at_register.c)
was supposed to stop on its own and didn't.
Adding a delay to the first kill -INT $mypid makes the problem
more obvious.
Stan Cox [Wed, 13 Oct 2021 16:43:35 +0000 (12:43 -0400)]
Support softfloat by dyninst backend.
The softfloat library has conflicting types for some types defined in
stdint.h, which -dyninst includes, so avoid by using cpp to redefine
them. Add a float test to sdt_types.c. Only test dyninst with
V3_uprobe. Add x8664 float registers to register definition used by
dyninst.
Frank Ch. Eigler [Tue, 12 Oct 2021 15:41:32 +0000 (11:41 -0400)]
BZ2012907 systemtap.spec: use sysuser.d/* for user/group management
Newer systemd systems (Fedora 32+) have an a declarative mechanism for
creation of system users/groups, instead of groupadd/useradd calls in
the %pre scripts. Switch to this scheme for modern enough systems.
William Cohen [Mon, 11 Oct 2021 20:26:06 +0000 (16:26 -0400)]
Update syscall_num.stp mappings between syscall number and name
Need to generate new versions of syscall_num.stp files that include
the preprocessor architecture guards so multiple syscall_num.stp
can be used by BPF code at the same time.
William Cohen [Mon, 11 Oct 2021 20:17:41 +0000 (16:17 -0400)]
Allow multiple architecture syscall_num.stp files to be used by bpf backend
Systemtap's BPF still has the concept of target architecture and isn't
write once run anywhere for BPF code. Thus, to make the syscall_any
tapset work the tapset needs to have all the different architecture
initialization code available and select the appropriate one based on
the architecture. Thus, each syscalls_num.stp file has preprocessing
guards to make them empty unless the architecture matches.
William Cohen [Mon, 11 Oct 2021 19:49:33 +0000 (15:49 -0400)]
Update dump-syscalls.sh to work with newer strace-code
There have been some changes in the strace-code:
-Header files now in strace-code/src/linux/*/syscallent.h
-mips headers use BASE_NR and file needs to be massaged to get numeric value
-corrected the name of riscv architecture name to riscv64
There are lots of races when printing warnings/errors/dbugs in staprun
because multiple eprintf() calls are used to print a single message and
stderr is not line-buffered. As a result, warnings/errors/dbugs race with
the relayfs reader threads printing to stdout and with other stap scripts
running concurrently in the same PTY. This causes the messages printed to
stderr and stdout to be garbled.
Fix all of this by using a single eprintf() for each warning/error/dbug
message, and by making stderr line-buffered so that we don't need to worry
about differing libc implementations potentially flushing a single message
in chunks rather than flushing the whole message in one go.
Sultan Alsawaf [Thu, 30 Sep 2021 02:42:26 +0000 (19:42 -0700)]
runtime: make _stp_vlog() more robust to avoid truncating log messages
Currently, _stp_vlog() very readily drops or truncates warnings, errors,
and debug messages. In the case of warnings and errors, this is quite
problematic because these messages are of high importance and, as such,
are even sent to stapio via the control channel rather than the relay
transport.
The reason why _stp_vlog() truncates and even drops these messages so
easily is twofold: the normal print buffer is used directly without any
attempt to flush it when there isn't enough space and it's used as
temporary storage for warnings and errors.
When warnings and errors are sent to the control channel, they are
copied into a new buffer, which is wasteful due to the copy operation
and the effort put into scrounging for space in the print buffer.
Instead of using a temporary buffer to construct warnings and errors,
it's more reliable and efficient to construct the message in one of the
control channel's buffers that would've been used anyway to send the
message.
In the case of debug messages, the print buffer can take appropriate
steps to ensure there's enough space via _stp_reserve_bytes(). Now, the
length of a debug message is calculated before it's generated, making it
possible to use _stp_reserve_bytes().
Altogether, this makes _stp_vlog() very resistant to losing both normal
debug messages and high-priority warning and error messages.
Stan Cox [Thu, 30 Sep 2021 20:11:29 +0000 (16:11 -0400)]
PR27829: Support floating point values passed through sdt.h markers
Add the type to the individual arg entries in the .notes.stapsdt section;
currently SP@A, where S is optional '-' sign, P is precision of type and A is
address. Revised format is SPT@A where T is optional 'f' for float variables.
Add x8664 float registers xmm8 - xmm15 and aarch64 float registers v8 - v31.
Parse the type field; result is currently ignored. asm statements are
restricted to 30 arguments; sdt probes can have up to 12 arguments. To fit
this into a single asm statement, precision and type are encoded into a single
field: 0xSSTT where SS is the precision and TT is the type as encoded by
__builtin_classify_type. The sign S, precision P, and type T are decoded by
_SDT_SIGN, _SDT_SIZE, and _SDT_TYPE. Test that the revised
.notes.stapsdt section interacts correctly with eu-elfutils and gdb.