]> sourceware.org Git - systemtap.git/log
systemtap.git
3 years agoconfigure.ac: use AM_PROG_AR to autodetect prefixed 'ar'
Sergei Trofimovich [Wed, 9 Sep 2020 01:08:04 +0000 (21:08 -0400)]
configure.ac: use AM_PROG_AR to autodetect prefixed 'ar'

Before the change
    ./configure --host=x86_64-pc-linux-gnu
    was using 'ar' tool to generate archives.

After the change it uses x86_64-pc-linux-gnu-ar tool.

It's useful for selecting one of multiple available
prefixed toolchains.

WARNING: Did not regenerate autotools files.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
3 years agoman stap.1: belatedly mention type auto-casting
Frank Ch. Eigler [Thu, 27 Aug 2020 17:05:43 +0000 (13:05 -0400)]
man stap.1: belatedly mention type auto-casting

Way back in 2014, starting with 0fb0cac98ad48, an auto-casting
facility was added.  This allows context-variable derived pointers to
retain their type information as the pointers are moved around inside
systemtap integer variables & expressions.  This was only documented
in the 2.6 release NEWS.  Now it's in the man page too.

3 years agostapbpf: fix module name
Sagar Patel [Fri, 21 Aug 2020 22:35:00 +0000 (18:35 -0400)]
stapbpf: fix module name

3 years agoPR26511: fix probe-condition synthetic begin{} probe compilability
Frank Ch. Eigler [Thu, 20 Aug 2020 17:20:36 +0000 (13:20 -0400)]
PR26511: fix probe-condition synthetic begin{} probe compilability

Fix two bugs:

1) In semantic_pass_optimize1, preserve synthetic probes even with
   empty bodies, which would normally be elided.  This allows
   probe-conditions to be more properly initialized, and
   initially-false probes to be disarmed early.

2) In c_unparser::emit_probe(), call emit_lock() based on the
   needs_global_locks() lockworthiness of the proper probe (the
   derived_probe being translated, rather than the other derived_probe
   whose conditions are being evaluated)

This lets the new lock-pushdown.* tests work with or without -u (now
tested).

4 years agoPR26296: lock pushdown optimization
Frank Ch. Eigler [Mon, 17 Aug 2020 19:49:35 +0000 (15:49 -0400)]
PR26296: lock pushdown optimization

Implements an algorithm to push lock/unlock operations downward in the
syntax tree, to just enclose the smallest possible region that deals
with global variables.  This means two common patterns run with much more
concurrency than before:

global a
probe foo {
  if (condition)
     { a++ }
  else
     { something_else() }
}

will only lock globals -if- the condition is true, so something_else()
would run unlocked.  Also:

global a
probe foo {
  if (a)
    { long_twisty_operation(); }
}

will unlock globals right after the condition is evaluated, so
long_twisty runs unlocked.  Previous behaviour is avilable with
--compatible=4.3.  New test case lock-pushdown.stp asserts locking
conditions throughout various relevant constructs.

4 years agotestsuite: have control_limits.stp trigger MAXNESTING as intended
Frank Ch. Eigler [Mon, 17 Aug 2020 18:37:59 +0000 (14:37 -0400)]
testsuite: have control_limits.stp trigger MAXNESTING as intended

Newer optimization sequencing requires some real work to be
done inside recursive functions to prevent their elision.

Reported-By: Martin Cermak <mcermak@redhat.com>
4 years agoat_var_unresolved.exp: Adapt to new error msg wording.
Martin Cermak [Mon, 17 Aug 2020 15:48:47 +0000 (17:48 +0200)]
at_var_unresolved.exp: Adapt to new error msg wording.

4 years agoPR26392: removed the "stable" flag from unwunding tapset functions
Yichun Zhang (agentzh) [Fri, 14 Aug 2020 20:56:20 +0000 (13:56 -0700)]
PR26392: removed the "stable" flag from unwunding tapset functions

By design, stable function calls should be lightweight ones like pid()
and target(). unwinding functions like ubacktrace() are kinda expensive.
And marking them as stable calls would make them get always evaluated no
matter what.

Thanks fche for the help here.

4 years agoprint formatting fixes for stapbpf rlim_t
Thorsten Glaser [Fri, 14 Aug 2020 16:09:45 +0000 (12:09 -0400)]
print formatting fixes for stapbpf rlim_t

Explicit %llu and (unsigned long long) is required on some platforms.

Based on: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968327

4 years agoPR26379: Formatting directive type fix.
Martin Cermak [Thu, 13 Aug 2020 06:39:48 +0000 (08:39 +0200)]
PR26379: Formatting directive type fix.

4 years agotranslator: probe-condition error handling
Frank Ch. Eigler [Fri, 7 Aug 2020 15:57:08 +0000 (11:57 -0400)]
translator: probe-condition error handling

Ensure generated code for runtime errors in probe condition
expressions have a proper exit label to jump to.

4 years ago--enable-http: add backward compatibility for pre-MHD_Result libmicrohttpd
Frank Ch. Eigler [Thu, 6 Aug 2020 17:34:59 +0000 (13:34 -0400)]
--enable-http: add backward compatibility for pre-MHD_Result libmicrohttpd

Prior to the API-breaking v0.9.71, libmicrohttpd used "int" for
callback results.  We have to be conditional with the switch.

4 years agoMake dtrace generated code work with LTO (take 2)
William Cohen [Wed, 5 Aug 2020 13:43:17 +0000 (09:43 -0400)]
Make dtrace generated code work with LTO (take 2)

LTO needs to know which variables might be accessed by code.  In sdt.h
there is a fair amount of assembly code which LTO cannot analyze.  As
a result LTO would assume that the semaphores variables for various
userspace probes were not accessed by the generated code.  The linking
would fail because LTO would optimize away the semphore variables
causing undefined references.

The fix adds the semaphore variables as input operands to the assembly
statements.  This gives the LTO analyzer information that the
variables are used within the assembly statements and should not be
removed.

4 years agoFix --enable-http build errors by always using MHD_Result
Stan Cox [Tue, 4 Aug 2020 18:35:21 +0000 (14:35 -0400)]
Fix --enable-http build errors by always using MHD_Result

Return MHD_Result instead of int for: get_key_values,
connection_info::postdataiterator, server::access_handler,
server::queue_response

4 years agoPR26307: rhel6 porting tweak redux
Frank Ch. Eigler [Fri, 31 Jul 2020 21:06:24 +0000 (17:06 -0400)]
PR26307: rhel6 porting tweak redux

(So many #if kernel version branches in this part of the code.)

4 years agoPR26307: rhel6 porting tweak
Frank Ch. Eigler [Fri, 31 Jul 2020 21:06:24 +0000 (17:06 -0400)]
PR26307: rhel6 porting tweak

(So many #if kernel version branches in this part of the code.)

4 years agojava/HelperSDT.c: correct 32-bit pointer cast warnings
Frank Ch. Eigler [Fri, 31 Jul 2020 20:59:33 +0000 (16:59 -0400)]
java/HelperSDT.c: correct 32-bit pointer cast warnings

Some casts through (uintptr_t) do the job.

4 years agoPR26307: adapt to kernel module_sect_attr changes in 5.8+
Frank Ch. Eigler [Fri, 31 Jul 2020 20:29:29 +0000 (16:29 -0400)]
PR26307: adapt to kernel module_sect_attr changes in 5.8+

Linux v5.8 rc5ish, commit ed66f991bb19, introduced a change to the
internal module_sect_attr types.  It caused a kernel BUG: with
pr14546.exp.  This showed it's futile to keep chasing it this
particular way in runtime/transport/symbols.c, so for new enough
kernels (4.7+), a method based on
kernel_read_file_from_path("/sys/module/$MODULE/sections/$SECTION") is
used to extract module section base-addresses.

This is done by a new paper-thin abstraction.  Tested on RHEL7, F32,
rawhide.

4 years agosystemtap.spec: let -testsuite subrpm require elfutils-debuginfod
Frank Ch. Eigler [Wed, 29 Jul 2020 18:45:28 +0000 (14:45 -0400)]
systemtap.spec: let -testsuite subrpm require elfutils-debuginfod

.... for sake of the sdt_buildid.exp test case

4 years agoPR26249: "%p" -> "0x%lx" pointer formatting in *conversions.stp error messages
Craig Ringer [Wed, 29 Jul 2020 00:28:07 +0000 (20:28 -0400)]
PR26249: "%p" -> "0x%lx" pointer formatting in *conversions.stp error messages

function::user_string() and function::user_string_warn() in uconversions.stp
were passing an "unsigned long" to the "%p" format-specifier, which upset gcc10.

Switch to 0x%lx. In the process, change other uses of %p to use 0x%lx instead,
changing casts from (void*) to (unsigned long) where necessary.
Fixes the following error on gcc10 (Fedora 32):

    In function ‘function___global_user_string_n_warn__overload_1’:
        /tmp/stapXXXXXX/stap_[...]_src.c: error: format ‘%lx’ expects
        argument of type ‘long unsigned int’, but argument 5 has type
        ‘void *’ [-Werror=format=]

Analogous fixes for kernel_string() etc.

4 years agoPR25568 / RHBZ1857749: sdt_buildid.exp test case
Frank Ch. Eigler [Mon, 27 Jul 2020 11:58:30 +0000 (07:58 -0400)]
PR25568 / RHBZ1857749: sdt_buildid.exp test case

Add new test that checks for combinations of buildid and pathname
based uprobes for executables and shared libraries.

4 years agoPR25568 / RHBZ1857749: buildid/uprobes/inode rework, task_finder etc. side
Frank Ch. Eigler [Mon, 27 Jul 2020 11:58:30 +0000 (07:58 -0400)]
PR25568 / RHBZ1857749: buildid/uprobes/inode rework, task_finder etc. side

During work on a new stress tests for build-id based probes (coming in
next commit), it was found that the task_finder2 logic for buildid
verification didn't, well, work, because it was never run (due to an
erroneous pathlen conditional), and couldn't be safely run where it
was (because it was under spinlock but would have done
access_process_vm).  Reworked the relevant bits of task_finder2 to
perform build-id verification for processes later - during the quiesce
callback periods.  (Buildid verification for solibs is already done
in the task_finder2 consumer uprobes-inode.c.)

Testing with sdt_misc indicated a case where a preexisting process
(with solib sdt.h semaphores) was being attached to by a new stap
binary.  task_finder2's enumeration of the preexising processes'
memory map segments violated assumptions by recent code related to
tracking in stapiu_process[] lists.  (It did not mirror the temporal
ld.so mmap sequence.)  Changed this tracking to use the inode* as the
key, and stop trying to track mapping lengths, to make positive
matches and eliminate duplicate stapiu_process[] entries for the same
(process,solib) permutation.  Reworked stapiu_process[] accumulation
generally to move to the two immediate task_finder callbacks, out of
stapiu_change_plus().

Added lots of commentary and diagnostics throughout.  stap
-DDEBUG_UPROBES give meaningful info about uprobes & sdt semaphores;
with -DDEBUG_TASK_FINDER, more but not overwhelming relevant info
appears.

4 years agoPR26234: stapbpf should warn about other unsupported derived_probe types
Serhei Makarov [Tue, 21 Jul 2020 15:00:20 +0000 (11:00 -0400)]
PR26234: stapbpf should warn about other unsupported derived_probe types

Again, print a warning instead of silently ignoring unsupported probes.

While a script may still be useful (in some cases) with only a subset
of the probes, it's probable the user may want to know about this.

4 years agotestsuite: shrink output
Frank Ch. Eigler [Sat, 18 Jul 2020 19:48:43 +0000 (15:48 -0400)]
testsuite: shrink output

The buildok/syscalls-detailed.stp test case spews several hundred
thousand lines of output.  Reduce verbosity from 4 to 2.

4 years agorhbz1857749: uprobes-inode regression in sdt semaphore setting
Frank Ch. Eigler [Sat, 18 Jul 2020 02:33:04 +0000 (22:33 -0400)]
rhbz1857749: uprobes-inode regression in sdt semaphore setting

Previous code neglected to set sdt.h semaphores for more than the
first process systemtap happened to encounter.  This was from a
mistaken understanding of what it meant for stapiu_change_plus() to be
called with the same inode/consumer combination.  Even though uprobes
are automatically shared, each new process still needs its perfctr and
sdt-semaphores individually set, so we do that now (as before the
rework of this code).  Mechanized testing incoming shortly.

4 years agoconfigury: make systemtap buildable with gcc -flto
Frank Ch. Eigler [Wed, 15 Jul 2020 22:34:33 +0000 (18:34 -0400)]
configury: make systemtap buildable with gcc -flto

Two problems corrected:

1 - Some of the stap object files compiled the interned_string type
    differently based on whether config.h was #included early enough
    or not.  Now they all do it at the top.

2 - The staprun/libstrfloctime.a pseudo-archive (added in commit
    0a5f4aa83e5b4 to work around an autoconf limitation), needs a
    -Wl,--whole-archive wrapper around it for lto linking purposes.

Now also stap builds on f32 both with and without:

  make CXX="g++ -flto" CC="gcc -flto"

4 years agoMake dtrace generated code work with LTO
William Cohen [Tue, 14 Jul 2020 20:36:24 +0000 (16:36 -0400)]
Make dtrace generated code work with LTO

LTO will attempt to remove variables if they are not clearly as used
in assembly code and the linking will fail with errors about undefined
variables. The semaphores variables in the sdt.h macros need to be
marked as global to ensure that LTO doesn't incorrect remove them.

4 years agoRevert "Make dtrace generated code work with LTO"
William Cohen [Tue, 14 Jul 2020 21:01:43 +0000 (17:01 -0400)]
Revert "Make dtrace generated code work with LTO"

This reverts commit 8212024da2ae6536a7d72f36cb688cb794cfe22e.

4 years agosystemtap.spec: comment syntax tweak
Frank Ch. Eigler [Tue, 14 Jul 2020 16:55:03 +0000 (12:55 -0400)]
systemtap.spec: comment syntax tweak

Move two warning-inducing # comments away from rpm %operator lines.

4 years agoMake dtrace generated code work with LTO
William Cohen [Mon, 13 Jul 2020 20:10:22 +0000 (16:10 -0400)]
Make dtrace generated code work with LTO

When variables are marked with the unused attribute LTO will attempt
to remove variables when they are only used by assembly code.  Marking
the semaphores variables generated by the dtrace with the used
attribute will ensure that LTO doesn't incorrect remove them.

4 years agoPR26234: stapbpf should warn about unsupported utrace_derived_probes
Serhei Makarov [Mon, 13 Jul 2020 17:36:54 +0000 (13:36 -0400)]
PR26234: stapbpf should warn about unsupported utrace_derived_probes

Probes such as process.begin are handled by task_finder, which is
not supported by BPF. Until some workaround is found, print
a warning instead of silently ignoring the probes.

4 years agotestsuite: block listing_mode_sanity large tests on small machine
Frank Ch. Eigler [Sun, 12 Jul 2020 19:07:01 +0000 (15:07 -0400)]
testsuite: block listing_mode_sanity large tests on small machine

Use a /proc/meminfo based heuristic to block two particularly
cpu/ram-costly tests on small machines, for same reasons as we nuked
semok/twenty.stp some time ago.

4 years agoRHBZ1847676 cont'd: one more uprobes-inode/onthefly concurrency control
Frank Ch. Eigler [Sun, 12 Jul 2020 13:57:15 +0000 (09:57 -0400)]
RHBZ1847676 cont'd: one more uprobes-inode/onthefly concurrency control

In uprobes-inode.c (stapiu_change_plus), the runtime can react to
arrivals of new mappings of a solib or executable by registering new
uprobes.  Due to an assumption that this could not happen at
inconvenient times (such as a stapiu_refresh or near shutdown times),
the actual uprobes registration operation was done outside the
consumer_lock mutex being held.  But it appears this can happen at bad
times, so the mutex needs to be held, just like within
stapiu_consumer_refresh().

The onthefly tests now survive iterating testing on rawhide+lockdep
and rhel7+lockdep.

4 years agoruntime: startup dmesg %p decloaking
Frank Ch. Eigler [Sun, 12 Jul 2020 12:11:45 +0000 (08:11 -0400)]
runtime: startup dmesg %p decloaking

On recent kernels, printk("...%p...") pointers are obfuscated by
default.  Tweak the systemtap startup dmesg to present a real pointer
(via "%lx"), which is essential for troubleshooting.

4 years agorun-stap.in: forget about bundled elfutils build mode
Frank Ch. Eigler [Sat, 11 Jul 2020 16:37:21 +0000 (12:37 -0400)]
run-stap.in: forget about bundled elfutils build mode

The "run-stap" script in the build tree allows developers to run
a freshly built, un-installed copy of systemtap.  It still knew of
our former bundled-elfutils build mode.  That has been removed,
so this logic is now gone from this script.

4 years agoPR25549: format tweak
Frank Ch. Eigler [Fri, 10 Jul 2020 03:10:53 +0000 (23:10 -0400)]
PR25549: format tweak

size_t likes printf("%zu")

4 years agoPR25549: statement probe visibility for openmp / lto binaries
Frank Ch. Eigler [Fri, 10 Jul 2020 02:47:36 +0000 (22:47 -0400)]
PR25549: statement probe visibility for openmp / lto binaries

gcc -flto and -fopenmp can synthesize function bodies that may miss
DWARF attributes such as AT_decl_file.  Previously, systemtap included
a check to match the value of that attribute against a list of file
names extracted from the source-file directory, and would reject any
function DIE that doesn't match.  So these -flto/-fopenmp functions
were invisible to systemtap probes.  We now override this for the case
for functions that completely lack that attribute, to expose these to
normal dwarfy probing.  New test case included (using -fopenmp).

4 years agotestuite: More @cast() fallout
Frank Ch. Eigler [Fri, 10 Jul 2020 01:41:51 +0000 (21:41 -0400)]
testuite: More @cast() fallout

Adjust another test case that uses the deprecated

   probe begin { @cast(PTR, "type") }

construct.  Now "kernel" is formally required to specify context.

4 years agoREADME: modernize
Frank Ch. Eigler [Fri, 10 Jul 2020 01:02:05 +0000 (21:02 -0400)]
README: modernize

Updated all URLs to https://.  Mentioned debuginfod as a viable
source of debuginfo.  Dropped mention of utrace.

4 years agoTapset and testsuite updates against @cast() change 00ee19ff03
Martin Cermak [Thu, 9 Jul 2020 07:19:01 +0000 (09:19 +0200)]
Tapset and testsuite updates against @cast() change 00ee19ff03

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to include kernel as location for
this information for the  @cast() rather than just assuming a default.

These are respective tapset and testsuite minor updates.

4 years agoRHBZ1847676 cont'd: more uprobes-inode/onthefly concurrency controls
Frank Ch. Eigler [Mon, 6 Jul 2020 17:27:46 +0000 (13:27 -0400)]
RHBZ1847676 cont'd: more uprobes-inode/onthefly concurrency controls

The systemtap.onthefly/*.exp tests had recently become hang-prone on
some kernels, for reasons still not completely understood.  This set
of patches adds:

- irq*-block spinlocks into uprobes-invoked paths, in case there is
  peculiar reentrancy (from irq-related tracepoints)

- a mutex lock/unlock into the stapiu_exit() path, in case there is
  a concurrent stapiu_refresh() invoked by onthefly machinery around
  exit time

- restrictions into the onthefly module_refresh() translator code to
  preclude STAP_SESSION_STOPPING as a time to do any sort of refresh
  operation.  Now probes that were disarmed will stay disarmed during
  probe-end/error/etc. processing, which is always valid with the
  spec, and avoids a class of late module-refresh ops

Testing on rhel7 and rawhide indicates the reproducible hang is gone.
Our testsuite already tortures this code; invoke by hand via:

% sudo make installcheck RUNTESTFLAGS="-v affection.exp hrtimer_onthefly.exp kprobes_onthefly.exp tracepoint_onthefly.exp uprobes_onthefly.exp"

4 years agotapset: *user_string* formatting directive tweaks
Frank Ch. Eigler [Thu, 2 Jul 2020 18:59:30 +0000 (14:59 -0400)]
tapset: *user_string* formatting directive tweaks

Some versions of gcc complain about an error-message
formatting mismatch:

'probe begin {println(user_string_n_warn(0, 20)) }'
->
/var/tmp/stapk2aFPs/stap_b9f8a6b29bbfa7f7e051c7587bbf7762_1907_src.c:288:40: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 5 has type ‘void *’ [-Werror=format=]
  288 |     "user string copy fault 0x%ld at %lx [man error::fault]", rc,

This switches to "0x%lx", (uintptr_t) consistently in related functions.

4 years agoPR26142: Adapt to linux/vermagic.h file hiding ... but with proper include order
Frank Ch. Eigler [Wed, 1 Jul 2020 00:17:14 +0000 (20:17 -0400)]
PR26142: Adapt to linux/vermagic.h file hiding ... but with proper include order

<linux/version.h> rock beats LINUX_VERSION_CODE scissors

4 years agoPR26142: Adapt to linux/vermagic.h file hiding ... but not on rhel6
Frank Ch. Eigler [Tue, 30 Jun 2020 23:36:17 +0000 (19:36 -0400)]
PR26142: Adapt to linux/vermagic.h file hiding ... but not on rhel6

Keep working on pre-3.17.0 kernels, which didn't have the
<generated/utsrelease.h> file yet.

4 years agoPR26142: adapt to linux mmap_sem api transition
Frank Ch. Eigler [Tue, 30 Jun 2020 23:04:16 +0000 (19:04 -0400)]
PR26142: adapt to linux mmap_sem api transition

With upstream linux 5.8-bound commit da1c55f1b27, the
mm_struct->mmap_sem field is renamed, and a new <mmap_lock.h> api is
introduced to lock/unlock it.  Adapting only the most recent runtime
parts (task_finder2.c and access_process_vm.h) to the new api by
providing autoconf-driven macro wrappers.  Other
backward-compatibility parts of the stap runtime still use mmap_sem,
but those are only compiled on old kernels, so don't appear to need
this porting.  Tested on 5.8.0-0.rc2.20200626git4a21185cda0f.1.fc33.x86_64
and 5.6.19-300.fc32.x86_64.

4 years agoPR26142: Adapt to linux/vermagic.h file hiding
Frank Ch. Eigler [Tue, 30 Jun 2020 19:24:18 +0000 (15:24 -0400)]
PR26142: Adapt to linux/vermagic.h file hiding

Linux commit 51161bfc66a68 (2020-04-19) enforces a convention of not
including <linux/vermagic.h> from normal source files.  Switch to
<generated/utsrelease.h>, which has UTS_RELEASE, which is all that we
really wanted anyway.

4 years agoPR26181: Use explicit @cast() within get_ip_from_client()
Martin Cermak [Mon, 29 Jun 2020 14:30:34 +0000 (16:30 +0200)]
PR26181: Use explicit @cast() within get_ip_from_client()

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to include kernel as location for
this information for the  @cast() rather than just assuming a default.

Also, fix the type of server_ip, which historically had been a long,
but since systemtap_v >= "4.3", it is a string.

4 years agoExclude some __NR_compat_* defines for newer aarch64 kernels
William Cohen [Wed, 24 Jun 2020 17:21:21 +0000 (13:21 -0400)]
Exclude some __NR_compat_* defines for newer aarch64 kernels

On aarch64 Linux system calls related SystemTap scripts
compilation fail with "__NR_compat_gettimeofday redefined".
errors due to:

    aarch64 kernel defines __NR_compat_gettimeofday after
     53c489e1dfeb arm64:  arm64: compat: Add missing syscall numbers

Fixed by adding proper conditional compilation based on current
architecture and kernel version.

4 years agoUse explicit @cast() operators pointing to kernel for tapsets
William Cohen [Mon, 22 Jun 2020 15:28:32 +0000 (11:28 -0400)]
Use explicit @cast() operators pointing to kernel for tapsets

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to include kernel as location for
this information for the  @cast() rather than just assuming a default.

4 years agoPR26131: garbled data might appear in staprun data channel output
Yichun Zhang (agentzh) [Mon, 22 Jun 2020 03:57:35 +0000 (20:57 -0700)]
PR26131: garbled data might appear in staprun data channel output

The relay v2 data writing lacked inode locking protection against
concurrent readers (on the stapio side).

The garbled data often contains '\0' bytes and might replace good data
or get appended to the good data stream.

Because the context of relay data writers may not allow sleeping, we use
a simple bounded spinlock to acquire the inode lock at our best effort
(up to about 300K cycles). Data loss might occur when we fail to obtain
the inode lock in which case it will be recorded as a "transportation
failure" and will get reported as a warning message. Increasing the
buffer size via 'staprun -b SIZE' would help reduce the risk of lock
contention.

The relay v1 implementation is not covered in this patch since it is old
and not even used on CentOS 6.

Succcessfully stress-tested on CentOS 6, CentOS 7, and Fedora 28. Also
tested with the kernel-debug packages' kernels on CentOS 7 and Fedora 28.

4 years agoUse kernel.trace("sched:sched_process_fork") for kprocess.create when possible
William Cohen [Thu, 18 Jun 2020 21:14:30 +0000 (17:14 -0400)]
Use kernel.trace("sched:sched_process_fork") for kprocess.create when possible

With optimization the copy_process function is often inlined making it
impossible for kprocess.create to probe the return of the copy_process
function.  The equivalent tracepoint sched:sched_process_fork should
be used instead to avoid that issue.  This change allows the
forktracker.stp and spawn_seekeer.stp examples to run even on kernels
where copy_process has been inlined.

4 years agoUse explicit @cast() operators for semop-watch.stp example.
William Cohen [Thu, 18 Jun 2020 19:01:40 +0000 (15:01 -0400)]
Use explicit @cast() operators for semop-watch.stp example.

4 years agoUse explicit @cast() operators for periodic.stp
William Cohen [Thu, 18 Jun 2020 17:32:50 +0000 (13:32 -0400)]
Use explicit @cast() operators for periodic.stp

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to include kernel as location for
this information for the  @cast() rather than just assuming a default.

4 years agoPR24758: increase stack size for BPF userspace interpreter
Serhei Makarov [Wed, 17 Jun 2020 19:48:56 +0000 (15:48 -0400)]
PR24758: increase stack size for BPF userspace interpreter

Requires parametrizing MAX_BPF_STACK by the target.

* bpf-internal.h (MAX_BPF_KERNEL_STACK): new constant.
(MAX_BPF_USER_STACK): new constant.
(MAX_BPF_STACK): change to a macro depending on target.
(program::use_tmp_space): pass target to MAX_BPF_STACK.

* bpf-opt.cxx (alloc_literal_str): pass target to MAX_BPF_STACK.
(spill): pass target to MAX_BPF_STACK.

* stapbpf/bpfinterp.cxx (bpf_interpret): increase stack size
to match MAX_BPF_USER_STACK in bpf-internal.h.

4 years agoUse explicit @cast() operators for stapgames/pingpong.stp tapset.
William Cohen [Wed, 17 Jun 2020 17:53:58 +0000 (13:53 -0400)]
Use explicit @cast() operators for stapgames/pingpong.stp tapset.

4 years agoUse explicit @cast() operators for pfiles.stp and ioctl_handler.stp
William Cohen [Wed, 17 Jun 2020 17:39:20 +0000 (13:39 -0400)]
Use explicit @cast() operators for pfiles.stp and ioctl_handler.stp

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to include kernel as location for
this information for the  @cast() rather than just assuming a default.

4 years agoRemove the unneeded test_support check the lwtools meta info
William Cohen [Wed, 17 Jun 2020 17:13:43 +0000 (13:13 -0400)]
Remove the unneeded test_support check the lwtools meta info

The test_support is to check to see if some needed feature or probe
point is available.  Using the example itself as the test for support
is not useful, so removing them.

4 years agoUse explicit @cast() operators to fslatency-nd.stp and fsslower-nd.stp
William Cohen [Wed, 17 Jun 2020 17:08:30 +0000 (13:08 -0400)]
Use explicit @cast() operators to fslatency-nd.stp and fsslower-nd.stp

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to include kernel as location for
this information for the  @cast() rather than just assuming a default.

4 years agoFix sizeof.stp to explicitly use kernel debuginfo if one not specified
William Cohen [Wed, 17 Jun 2020 15:57:18 +0000 (11:57 -0400)]
Fix sizeof.stp to explicitly use kernel debuginfo if one not specified

Commit 00ee19ff030f665df7e087a579f39105256a0253 changed how @cast()
operations work and they no longer default to using the kernel
debuginfo for type information.  Need to use the @cast_module_sizeof()
instead of @cast_size() to use the kernel debuginfo.

4 years agobpf.exp: tentative fix for bigmap1.stp hang on RHEL8
Serhei Makarov [Wed, 17 Jun 2020 14:21:10 +0000 (10:21 -0400)]
bpf.exp: tentative fix for bigmap1.stp hang on RHEL8

Experimentation with the prior bigmap1.stp patch
shows that bpf.exp may need a stronger kill -KILL to
avoid waiting for a stuck process.

Not sure why this wasn't done in the first place,
the comment suggests kill -KILL but the code does kill -INT.

4 years agoRHBZ1847676: uprobes-inode tweaks redux
Frank Ch. Eigler [Wed, 17 Jun 2020 00:35:53 +0000 (20:35 -0400)]
RHBZ1847676: uprobes-inode tweaks redux

Added (back) a spinlock to manage the stapiu_consumer -> process_list
structure, since it is occasionally travered from uprobe pre-handlers,
which are sometimes entered in atomic context (e.g. on rhel7).  There,
the normal mutex_t is unsafe.  So restoring a spinlock_t just for
those shortlived traversals, rhel7 and rawhide are both happy.

4 years agoMake sizeof.stp runnable with the bpf backend.
William Cohen [Tue, 16 Jun 2020 20:02:11 +0000 (16:02 -0400)]
Make sizeof.stp runnable with the bpf backend.

4 years agoremove comment
Sagar Patel [Thu, 11 Jun 2020 14:37:46 +0000 (10:37 -0400)]
remove comment

4 years agoPR26123: fixed memory leaks in vma map for kernels without CONFIG_UTRACE.
Yichun Zhang (agentzh) [Tue, 16 Jun 2020 18:10:13 +0000 (11:10 -0700)]
PR26123: fixed memory leaks in vma map for kernels without CONFIG_UTRACE.

4 years agoAdd overcommit.stp to examples that can run in bpf.
William Cohen [Tue, 16 Jun 2020 14:12:54 +0000 (10:12 -0400)]
Add overcommit.stp to examples that can run in bpf.

4 years agoconfigury: post-release version bump
Frank Ch. Eigler [Mon, 15 Jun 2020 20:40:28 +0000 (16:40 -0400)]
configury: post-release version bump

4 years agoAvoid exceeding space constraints for the BPF environment in mmfilepage.stp
William Cohen [Thu, 11 Jun 2020 20:19:26 +0000 (16:19 -0400)]
Avoid exceeding space constraints for the BPF environment in mmfilepage.stp

The BPF runtime environment is very space constraint.  The
mmfilepage.stp example has been reorganized to reduce the stack space
and number of BPF registers needed to run the code to allow the
generated BPF code to fit within the constraints.

4 years agoAvoid exceeding space constraints for BPF environment in mmwriteback.stp
William Cohen [Thu, 11 Jun 2020 19:54:32 +0000 (15:54 -0400)]
Avoid exceeding space constraints for BPF environment in mmwriteback.stp

BPF is very contrained on the space allowed when running the code.
Using too complicated printf statements caused too much stack space to
be used for temporaries and the stap compile would fail for the BPF
backend.  Replacing the complicated printf with simpler string printf
statements uses less stack space and allows the BPF backend to compile
this example.

4 years agoprerelease: update-docs release-4.3
Frank Ch. Eigler [Thu, 11 Jun 2020 14:35:51 +0000 (10:35 -0400)]
prerelease: update-docs

4 years agodocs generator: tweak for python3
Frank Ch. Eigler [Thu, 11 Jun 2020 14:35:28 +0000 (10:35 -0400)]
docs generator: tweak for python3

4 years agoprerelease: PRERELEASE source bumps
Frank Ch. Eigler [Thu, 11 Jun 2020 14:26:16 +0000 (10:26 -0400)]
prerelease: PRERELEASE source bumps

4 years agoprerelease: AUTHORS bump
Frank Ch. Eigler [Thu, 11 Jun 2020 14:23:41 +0000 (10:23 -0400)]
prerelease: AUTHORS bump

4 years agoprerelease: update-po
Frank Ch. Eigler [Thu, 11 Jun 2020 14:23:23 +0000 (10:23 -0400)]
prerelease: update-po

4 years agotapset: fix ctime.stp handling of STAPCONF_TIME32_OLD_H
Frank Ch. Eigler [Thu, 11 Jun 2020 14:00:04 +0000 (10:00 -0400)]
tapset: fix ctime.stp handling of STAPCONF_TIME32_OLD_H

We need a type alias that works for ctime.stp even if the #defines
in aux_syscalls.stp were not already pulled in.

4 years agobpf-translate.cxx: avoid dereferencing empty vectors
Frank Ch. Eigler [Thu, 11 Jun 2020 13:50:00 +0000 (09:50 -0400)]
bpf-translate.cxx: avoid dereferencing empty vectors

Under some conditions, a bpf translation-pass vector could be read
empty.  We check better.

4 years agoPR26019 stapbpf: don't continue startup if begin probe already exited
Serhei Makarov [Thu, 11 Jun 2020 00:22:06 +0000 (20:22 -0400)]
PR26019 stapbpf: don't continue startup if begin probe already exited

if a bpf begin probe sets exit status, the main thread will skip the
pause() call and immediately deallocate global data structures. Then
the perf_event_loop thread will run on the corrupted data and
occasionally fail. Example result is an assertion failure due
count_active_cpus() running on corrupted data.

Could change the main thread to join() instead of detach(), but then
all threads must be modified to listen for exit status including exit
status from begin thread. TODO Consider doing so later.

I think if the begin probe was running, it wasn't correct to start the
perf_event_loop (or PERF_EVENT_IOC_ENABLE, or any of the procfs
threads) in the first place, as those things logically happen after
the begin probe has already exited. Simply clean up and exit.

4 years agouprobes-inode: rework for buildid vs. onthefly work
Frank Ch. Eigler [Thu, 11 Jun 2020 00:28:01 +0000 (00:28 +0000)]
uprobes-inode: rework for buildid vs. onthefly work

Tweak locking in the cleanup paths. stapiu_consumer_unreg (at
stap exit time) now does all cleanup, including its process_list,
and stapiu_change_minus (at probed process exit) does nothing.
This seems to strike a good balance between cheering up lockdep
and using a little more memory during very long-lived stap jobs.

4 years agouprobes-inode: rework for buildid vs. onthefly work
Frank Ch. Eigler [Mon, 8 Jun 2020 13:28:06 +0000 (09:28 -0400)]
uprobes-inode: rework for buildid vs. onthefly work

Addition of the buildid-based probes has highlighted some weaknesses
in the uprobes-inode code with respect to the interlinking & sharing
of various tracking data structures.  This patch reworks the code to
simplify and document.  The uprobes_onthefly.exp test case easily
survives now.

4 years agogcc compatibility: use gcc #pragma to suppress -Wtautological-compare
Frank Ch. Eigler [Mon, 8 Jun 2020 13:26:35 +0000 (09:26 -0400)]
gcc compatibility: use gcc #pragma to suppress -Wtautological-compare

On some compilers / distros, the kbuild-flavour cc-options runtime
test for suppressing -Wtautological-compare does not work.  So we
switch to a pragama based suppression that works as far back as rhel7
gcc 4.8.5.  We need to suppress that warning because script-originated
generated code can easily include tautologies.

4 years agoPR26074 fixup: disable #else paths on !USE_KALLSYMS_ON_EACH_SYMBOL
Serhei Makarov [Wed, 10 Jun 2020 17:47:07 +0000 (13:47 -0400)]
PR26074 fixup: disable #else paths on !USE_KALLSYMS_ON_EACH_SYMBOL

These previously #else paths should only be taken if
USE_KALLSYMS_ON_EACH_SYMBOL does not hold, not if
USE_KALLSYMS_ON_EACH_SYMBOL is true but the runtime condition is
false.

4 years agosession.cxx prerelease: declare support for kernel 5.7
Frank Ch. Eigler [Tue, 9 Jun 2020 19:09:54 +0000 (15:09 -0400)]
session.cxx prerelease: declare support for kernel 5.7

4 years agoPR26099: adapt to struct proc_ops split from struct file_operations
Serhei Makarov [Tue, 9 Jun 2020 16:31:34 +0000 (12:31 -0400)]
PR26099: adapt to struct proc_ops split from struct file_operations

Linux kernel commit d56c0d45f0e for version 5.6 made the following changes:
- procfs functions now take struct proc_ops instead of struct file_operations
- proc_dir_entry now has union{proc_ops,proc_dir_ops} instead of proc_fops

struct proc_ops is newly declared in the commit, so we test for its presence.

* buildrun.cxx (compile_pass): Introduce STAPCONF_PROC_OPS.
* runtime/linux/autoconf-proc-ops.c: New file. Test for struct proc_ops.

* runtime/procfs-probes.c (_stp_proc_fops): Declare as proc_ops when available.
* runtime/procfs.c (_stp_create_procfs): Take fops as proc_ops when available.
* runtime/transport/control.h (_stp_ctl_proc_ops.cmd): New proc_ops version of
_stp_ctl_fops_cmd for use
* runtime/transport/control.c (_stp_ctl_proc_ops.cmd): Ditto.
* runtime/transport/procfs.c (_stp_proc_fops): Declare as proc_ops when available.
(_stp_register_ctl_channel): struct proc_dir_entry renamed proc_fops to proc_ops,
adapt in two locations.

4 years agoman error::dwar7: mention kbuild -fno-var-tracking-assignments
Frank Ch. Eigler [Tue, 9 Jun 2020 13:59:00 +0000 (09:59 -0400)]
man error::dwar7: mention kbuild -fno-var-tracking-assignments

"The simplest fix is to erase, excise, nay, eradicate this"

4 years agoAdd probes to container_check.stp for additional capabilities checks
William Cohen [Tue, 9 Jun 2020 13:29:46 +0000 (09:29 -0400)]
Add probes to container_check.stp for additional capabilities checks

Probe points were added for ns_capable_noaudit, ns_capable_setid,
and file_ns_capable functions.  Before adding these probes some
capability checks were missed by the container_check.stp script.
This was observed running the script on DPDK testpmd.

4 years agoBug: __stp_tf_quiesce_worker(): vma mmap callback incorrecty ran after process.begin...
Yichun Zhang (agentzh) [Mon, 8 Jun 2020 22:22:33 +0000 (15:22 -0700)]
Bug: __stp_tf_quiesce_worker(): vma mmap callback incorrecty ran after process.begin callback

4 years agofeature: vma-related primitives now work in process.begin.
Yichun Zhang (agentzh) [Mon, 8 Jun 2020 00:10:15 +0000 (17:10 -0700)]
feature: vma-related primitives now work in process.begin.

4 years agoBug: tapset functions reading userland registers did not work in probe process.begin
Yichun Zhang (agentzh) [Sun, 7 Jun 2020 21:19:46 +0000 (14:19 -0700)]
Bug: tapset functions reading userland registers did not work in probe process.begin

4 years agoPR26074: pass kallsyms_lookup_name as a relocation to the stap runtime
Serhei Makarov [Fri, 5 Jun 2020 19:12:21 +0000 (15:12 -0400)]
PR26074: pass kallsyms_lookup_name as a relocation to the stap runtime

Kernel commit 0bd476e6c671 unexports kallsyms_lookup_name
and kallsyms_on_each_symbol.
As kallsyms_lookup_name is used to look up unexported symbols, we need to
use the relocation mechanism (the same one used to pass _stext) to pass
these symbols to the runtime where necessary.

The patch preserves the old behaviour (link the exported symbol directly)
where kallsyms_lookup_name is available.

* buildrun.cxx (compile_pass): Change output_autoconf
STAPCONF_KALLSYMS_ON_EACH_SYMBOL to exportconf
STAPCONF_KALLSYMS_ON_EACH_SYMBOL_EXPORTED. Since the symbol may be
accessed even when unexported, the autoconf-kallsyms-on-each-symbol.c
is no longer accurate. Control the powerpc issue (PR11514) with
appropriate #ifdefs instead.
Change STAPCONF_KALLSYMS to STAPCONF_KALLSYMS_LOOKUP_NAME_EXPORTED.
* runtime/linux/autoconf-kallsyms-on-each-symbol.c: Deleted.

* runtime/linux/runtime.h (_stp_kallsyms_lookup_name): New field.
(_stp_kallsyms_on_each_symbol): New field.
* runtime/sym.c (kallsyms_lookup_name_fn): New typedef.
(kallsyms_lookup_name): New function, linked in place
of kallsyms_lookup_name, calls _stp_kallsyms_lookup_name if available.
(kallsyms_on_each_symbol_fn): New typedef.
(kallsyms_on_each_symbol): New function, linked in place of
kallsyms_on_each_symbol, calls _stp_kallsyms_on_each_symbol if available.

* runtime/sym.h (_stp_need_kallsyms_stext): New field.
(_stp_set_stext): New function, called to adjust _stp_kretprobe_trampoline
if kallsyms_lookup_name("_stext") call is delayed until after the relocation
is received.
* runtime/transport/symbols.c (_stp_set_stext): New function.
(_stp_do_relocation): Set _stp_kretprobe_trampoline if kallsyms_lookup_name
is not yet available,
Accept and set _stp_kallsyms_lookup_name and _stp_kallsyms_lookup_symbol
from relocations.
* runtime/transport/transport.h (_stp_handle_kallsyms_lookups): New function,
does delayed kallsyms_lookup_name() lookups previously done
in _stp_transport_init(), does delayed _stext lookup.
* runtime/transport/transport.c: Include sym.h for _stp_set_stext().
(_stp_handle_kallsyms_lookups): New function.
(_stp_transport_init): Don't do kallsyms_lookup_name() lookups yet
as kallsyms_lookup_name may be passed later on via relocation.
Init the addresses and _stp_need_kallsyms_stext to NULL instead.
* translate.cxx (c_unparser::emit_module_init): Emit a call to
_stp_handle_kallsyms_lookups() into systemtap_module_init(),
where the relocations for kallsyms_lookup_name() will be available.
* staprun/staprun.c (send_relocation_kernel): Look up
kallsyms_lookup_name and kallsyms_on_each_symbol in /proc/kallsyms
and send their addresses as STP_RELOCATION messages.

* runtime/linux/kprobes.c: define USE_KALLSYMS_ON_EACH_SYMBOL.
May be determined statically or dynamically depending on whether
a relocation is necessary to call kallsyms_on_each_symbol.
(stapkp_prepare_kprobe): change #ifdef to if(USE_KALLSYMS_ON_EACH_SYMBOL).
(stapkp_prepare_kretprobe): Ditto.
(stapkp_symbol_data): make available in all circumstances.
(stapkp_init): change #ifdef to if(USE_KALLSYMS_ON_EACH_SYMBOL).
(stapkp_refresh): Ditto.

* runtime/transport/transport_msgs.h (enum): clarifying comment
for STP_RELOCATION message.

4 years agoconfigury: adjust -lebl check to ebl_strtabinit
Frank Ch. Eigler [Fri, 5 Jun 2020 16:16:19 +0000 (12:16 -0400)]
configury: adjust -lebl check to ebl_strtabinit

While -lebl has been integrated into -ldw in recent versions of
elfutils, some older versions need a better aimed check, for
the exact symbol that bpf-translate.cxx uses.

4 years agomove the new added blurb under version 4.3
Alice Zhang [Thu, 4 Jun 2020 18:36:19 +0000 (14:36 -0400)]
move the new added blurb under version 4.3

4 years agoupdate blurb related to PR13793
Alice Zhang [Thu, 4 Jun 2020 17:45:13 +0000 (13:45 -0400)]
update blurb related to PR13793

4 years agoRHBZ1842866 kernel-rt port: treat CONFIG_PREEMPT_RT as if ..._FULL
Juri Lelli [Wed, 3 Jun 2020 13:19:26 +0000 (09:19 -0400)]
RHBZ1842866 kernel-rt port: treat CONFIG_PREEMPT_RT as if ..._FULL

Until recently CONFIG_PREEMPT_RT_FULL was the config switch used by
PREEMPT_RT (and RHEL-RT up to 8.2) for building an RT kernel.
Upstream commit a50a3f4b6a313 ("sched/rt, Kconfig: Introduce
CONFIG_PREEMPT_RT") changed that to be CONFIG_PREEMPT_RT.

4 years agorename nfs_proc-embedded-newver with specific version indicated
Alice Zhang [Mon, 1 Jun 2020 18:42:34 +0000 (14:42 -0400)]
rename nfs_proc-embedded-newver with specific version indicated

4 years agoFix PR13793
Alice Zhang [Mon, 1 Jun 2020 17:44:41 +0000 (13:44 -0400)]
Fix PR13793

* tapset/linux/nfs_proc.stp: fix get_ip_from_client typo

4 years agoFix PR13793 by adding IPv6 support nfs_proc.stp
Alice Zhang [Mon, 1 Jun 2020 16:10:30 +0000 (12:10 -0400)]
Fix PR13793 by adding IPv6 support nfs_proc.stp

* tapset/linux/nfs_proc.stp: Updated get_ip_from_client and get_ip with IPv6 support for version after 4.3
* testsuite/buildok/nfs_proc-embedded.stp: Fix probe point so that embedded-C would work properly. Support version before 4.2.
* testsuite/buildok/nfs_proc-embedded-newver.stp: nfs_proc-embedded test case that supports version after 4.3.
* testsuite/buildok/rpc-embedded.stp: Fix probe point so that embedded-C would work properly.

4 years agoupdate foreach loop test
Sagar Patel [Mon, 1 Jun 2020 14:38:25 +0000 (10:38 -0400)]
update foreach loop test

4 years agoruntime/tapset: adapt to more time32 kernel deprecation
Frank Ch. Eigler [Sat, 30 May 2020 18:15:29 +0000 (14:15 -0400)]
runtime/tapset: adapt to more time32 kernel deprecation

Linux commit 412c53a6 et al., deprecate various 32-bit time-related
types/structs.  Many are replaced by userspace-compatibility types
under different names ("old_*") in a different header, so we #include
that.  In order to keep the runtime code as unchanged as possible, we
add some of the name-mapping macros into our own embedded-C code.

Tested on f31, rhel7.

4 years agotestsuite abort.exp: adapt to tweaked error message
Frank Ch. Eigler [Sat, 30 May 2020 13:51:57 +0000 (09:51 -0400)]
testsuite abort.exp: adapt to tweaked error message

Unresolved-function semantic errors now print something extra.

4 years agosyscall tapset: _do_fork args conversion
Frank Ch. Eigler [Thu, 28 May 2020 20:58:28 +0000 (16:58 -0400)]
syscall tapset: _do_fork args conversion

As of linux 5.3 commit 7f192e3cd316ba, the _do_fork() function probed
for syscall.clone switched its API to carry arguments curried in a
struct.  Adjusting the syscall.clone tapset to accept either API
via the @choose_defined() macro.

4 years agowarnings: don't complain about cross-file global var access within tapset
Frank Ch. Eigler [Thu, 28 May 2020 18:46:38 +0000 (14:46 -0400)]
warnings: don't complain about cross-file global var access within tapset

The syscall.* tapset uses this construct

tapset/linux/syscalls_cfg_trunc.stp:global syscall_string_trunc = 50

to configure pretty-printing.  There's no need to warn about
unintentional cross-file global var accesses in this case.
So the warning is restricted to tapsets matching with user-script
globals only.

This page took 0.068091 seconds and 5 git commands to generate.