]> sourceware.org Git - systemtap.git/log
systemtap.git
5 years agoconfigure: Allow explicit configuration of the monitor
Ross Burton [Thu, 5 Sep 2019 13:41:52 +0000 (14:41 +0100)]
configure: Allow explicit configuration of the monitor

Add an option to explicitly disable the monitor (and therefore the dependency on
json-c and ncurses) with the aim of deterministic builds.

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
5 years agoconfigure: Allow to disable libvirt
Wenzong Fan [Thu, 5 Sep 2019 13:41:51 +0000 (14:41 +0100)]
configure: Allow to disable libvirt

Rather than just autodetecting libvirt allow it to be explictly disabled even
if present (important for reproducibile builds).

Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
5 years agoruntime: 5.2 and later kernels removed get_ds() on arm and other architectures
Richard Purdie [Thu, 5 Sep 2019 13:13:13 +0000 (09:13 -0400)]
runtime: 5.2 and later kernels removed get_ds() on arm and other architectures

https://github.com/torvalds/linux/commit/736706bee3298208343a76096370e4f6a5c55915

Add in a fallback so systemtap works on arm in these cases.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
5 years agobuildrun: adapt to loss of "flags" filter in linux scripts/Kbuild.include
Frank Ch. Eigler [Wed, 4 Sep 2019 18:03:43 +0000 (14:03 -0400)]
buildrun: adapt to loss of "flags" filter in linux scripts/Kbuild.include

This $(call filter,...) was used since 2008 to pass KBUILD_CFLAGS
through a mechanism that preserved O= data somehow.  But this
filter is gone from the kernel as of commit cdd750bf, so add a
fallback for the filtered $KBUILD_CFLAGS directly.

Reported-By: RP on #systemtap
5 years agoPR24947, PR24946: check for valid hex and octal sequences
Sagar Patel [Thu, 29 Aug 2019 15:44:30 +0000 (11:44 -0400)]
PR24947, PR24946: check for valid hex and octal sequences

Hex escape sequences no longer causes scripts to crash on stapbpf,
and checks have been added to ensure valid sequence ranges.

1) Fixed incorrect looping of hex sequences.
2) Added octal sequence range check: o < \400.
3) Added hex sequence range check: h < \x100.

5 years agoPR23285 (2): enable prometheus-exporter type scripts on stapbpf
Sagar Patel [Thu, 29 Aug 2019 15:40:21 +0000 (11:40 -0400)]
PR23285 (2): enable prometheus-exporter type scripts on stapbpf

The eBPF backend now supports prometheus-exporter scripts. This implementation
introduces character escaping macros enabling the array dump macros on stapbpf.
However, there are some issues with foreach loops which are used in the macros
(PR24953).

1) Developed character escaping macros helper functions.
2) Added prometheus probe tapset for stapbpf.
3) Added stapbpf procfs file path to stap-exporter.
4) Introduced a sample stapbpf prometheus-exporter script in EXAMPLES.
5) Updated NEWS.

5 years agoPR24926: correct printing of utf-8 characters on stapbpf
Sagar Patel [Thu, 29 Aug 2019 15:32:17 +0000 (11:32 -0400)]
PR24926: correct printing of utf-8 characters on stapbpf

There were two bugs corrupting the string bytes and instructions. The
first bug involved the implicit sign extension of negative char values.
The second bug involved a faulty optimization (fixup_operands) which
used the incorrect instruction opcode.

1) Cast char to unsigned char before casting to uint32_t.
2) Changed opcode of optimized instruction to (BPF_STX | BPF_MEM | BPF_W).

5 years agoenable concatenation operator for stapbpf userspace probes
Sagar Patel [Thu, 29 Aug 2019 15:30:05 +0000 (11:30 -0400)]
enable concatenation operator for stapbpf userspace probes

5 years agokernel tracepoints: support afs, rxrpc, siox tracepoints for kernel 5.2
Frank Ch. Eigler [Sat, 31 Aug 2019 20:24:42 +0000 (16:24 -0400)]
kernel tracepoints: support afs, rxrpc, siox tracepoints for kernel 5.2

Added the usual forward-declarations or such for incomplete
tracepoint header files into tracepoint_extra_decls().

5 years agorhbz1731333: nfsd.proc.write, tolerate awol 'vlen'
Frank Ch. Eigler [Thu, 29 Aug 2019 22:18:59 +0000 (18:18 -0400)]
rhbz1731333: nfsd.proc.write, tolerate awol 'vlen'

This variable has disappeared from the kernel via commit
8154ef2776aa512a3eaa0e7db030dc4803354d61.  Substitute a 0.

5 years agodyninst configury: require 10.1+ but permit ppc64le and aarch64
Frank Ch. Eigler [Wed, 28 Aug 2019 12:31:02 +0000 (08:31 -0400)]
dyninst configury: require 10.1+ but permit ppc64le and aarch64

Modern dyninst works okay on ppc64le and aarch64.

5 years agostapbpf: correct 32-bit handling of EXIT bpf-map value type
Frank Ch. Eigler [Fri, 23 Aug 2019 23:23:13 +0000 (19:23 -0400)]
stapbpf: correct 32-bit handling of EXIT bpf-map value type

The bpf bytecode declares a 4byte->8byte map for the shared globals;
the userspace must use a 64-bit type to receive those values, lest
it suffer a most unfortunate stack smash.

5 years agotestsuite procfs_bpf.exp: tolerate stapbpf early exit
Frank Ch. Eigler [Fri, 23 Aug 2019 15:26:06 +0000 (11:26 -0400)]
testsuite procfs_bpf.exp: tolerate stapbpf early exit

It was observed on some platforms that the stap/bpf test case could
abort so quickly that the dejagnu (tclsh) code to open the bpf/procfs
pipes could itself hang - and indefinitely!  Replaced these tclsh
level ops with a /bin/timeout-wrapped echo or cat operation.

5 years agoPR23879, PR24875: fix task-finder-vma on f29+
Frank Ch. Eigler [Wed, 21 Aug 2019 23:29:45 +0000 (19:29 -0400)]
PR23879, PR24875: fix task-finder-vma on f29+

It was reported & rediscovered that some vma-dependent runtime
facilities have been broken: @vma() and *ubacktrace().  It turns out
that modern gcc/ld.so links/loads binaries in slightly different ways
than older toolchains.  Specifically, the first page of ELF files is
now loaded only r--p instead of r-xp protection flags.  The
_stp_vma_mmap_cb() routine now accepts the r--p case too.  It now
ignores the flags entirely.

5 years agoPR24904: support linux 5.2's stacktrace.c changes with -DDEBUG_UNWIND too
Frank Ch. Eigler [Wed, 21 Aug 2019 02:17:02 +0000 (22:17 -0400)]
PR24904: support linux 5.2's stacktrace.c changes with -DDEBUG_UNWIND too

5 years agoPR24904: support linux 5.2's stacktrace.c changes
Frank Ch. Eigler [Wed, 21 Aug 2019 01:20:40 +0000 (21:20 -0400)]
PR24904: support linux 5.2's stacktrace.c changes

The following kernel commit disabled the older struct stack_trace APIs
on architectures that support the newer stackwalk APIs.  Provide an
adaptation layer to stack_trace_save_regs().

commit 214d8ca6ee854f696f75e75511fe66b409e656db
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Thu Apr 25 11:45:21 2019 +0200

    stacktrace: Provide common infrastructure

5 years agostapbpf PR23858 :: support sorting by value in foreach loop
Serhei Makarov [Thu, 15 Aug 2019 15:09:06 +0000 (11:09 -0400)]
stapbpf PR23858 :: support sorting by value in foreach loop

Previously stapbpf's behaviour on this was wrong.
This slightly nightmarish bit of code handles more cases.
XXX Also need to take into account s->sort_aggr for statistics aggregates
-- currently unsupported.

To support nested foreach loops, map_get_next_key was changed to take & return
addresses-of string addresses rather than string addresses. Previously
we'd need to allocate and copy multiple string keys onto one tiny stack,
which is not feasible.

* bpf-internal.h (SORT_FLAGS): New define packs sort_column, sort_direction in one word.
  (GET_SORT_COLUMN): New define unpacks sort_column from SORT_FLAGS.
  (GET_SORT_DIRECTION): New define unpacks sort_direction from SORT_FLAGS.
* bpf-translate.cxx (bpf_unparser::visit_foreach_loop): String key/value iteration,
  hardcode keysize on stack to 8bytes (either an int or an address of a string),
  check for sort_aggr (UNSUPPORTED),
  pack sort_column and sort_direction into sort_flags,
  properly handle returned address of a string.
* stapbpf/bpfinterp.cxx (map_int_keys, map_str_keys): Remove typedefs.
  (struct map_keys): Nightmare begins. Support 4 cases of int/str keys/values.
  (convert_int_key, convert_str_key, convert_int_kp, convert_str_kp): More nightmare,
  handle different cases of packing/unpacking BPF key/val into C++ key/val.
  (convert_key, convert_kp): More nightmare. 'Overload' pack/unpack for ANY key/val.
  (compute_key_size): More nightmare. Calculate memory required for a key.
  (map_sort): New function handles initial sorting of map.
  (map_next): New function handles retrieval from sorted map.
  (map_get_next_key): Handle sorted int/str keys/values on different columns,
  now takes strings to give a place to allocate returned str key/vals
  that is not on the BPF stack.
  (bpf_intepret): Pass strings to map_get_next_key.

* testsuite/systemtap.bpf/bpf_tests/foreach_string.stp: Enable full testcase.
* testsuite/systemtap.bpf/bpf_tests/foreach_pr23858.stp: New testcase.

5 years agobpf_translate.cxx :: fix another segfault in visit_delete_statement
Serhei Makarov [Thu, 15 Aug 2019 15:02:59 +0000 (11:02 -0400)]
bpf_translate.cxx :: fix another segfault in visit_delete_statement

5 years agostapbpf: 32-bit build tweaks
Frank Ch. Eigler [Tue, 13 Aug 2019 19:36:25 +0000 (15:36 -0400)]
stapbpf: 32-bit build tweaks

On 32-bit hosts, uint64_t != unsigned long .

5 years agoPR23285: stapbpf/procfs: use effective-userid consistently
Frank Ch. Eigler [Tue, 13 Aug 2019 19:08:41 +0000 (15:08 -0400)]
PR23285: stapbpf/procfs: use effective-userid consistently

The euid is easy and consistent in the construction of the
stapbpf/procfs path name.  Use the umask() field at least for
the fifo nodes (should also on the parent directory probably).
For the stapbpf -v case, print some fifo-related diagnostics.
Split out the .stp test case into a separate file, for easier
hand-execution.

5 years agoexpliclity show pass3 for -v on bpf runtime
Sagar Patel [Fri, 9 Aug 2019 21:20:22 +0000 (17:20 -0400)]
expliclity show pass3 for -v on bpf runtime

5 years agoPR23285 (1): enable procfs probes for stapbpf
Sagar Patel [Thu, 8 Aug 2019 19:40:10 +0000 (15:40 -0400)]
PR23285 (1): enable procfs probes for stapbpf

The eBPF backend now supports procfs probes. This implementation
uses FIFO special files instead of proc filesystem files. The file
path format used is /var/tmp/systemtap-USER/MODNAME. One limitation
is that both read and write probes cannot exist for the same file.

1) Added procfs probe data structures to hold probe information.
2) Created an interface between target variables and eBPF interpreter.
3) Dedicated a single thread for each file which monitors for I/O.
4) Developed a cleaning routine and error handling mechanisms.
5) Updated NEWS and man pages.

small fix

new procfs_bpf test

5 years agorhbz1737589: add /* fallthru */ to satisfy gcc 9's -Wimplicit-fallthrough
Frank Ch. Eigler [Sun, 11 Aug 2019 15:00:27 +0000 (15:00 +0000)]
rhbz1737589: add /* fallthru */ to satisfy gcc 9's -Wimplicit-fallthrough

We need annotate switch statements with deliberate case-to-case fallthroughs.

5 years agoAdd example using the test_check_dyninst and test_installcheck_dyninst.
William Cohen [Wed, 7 Aug 2019 14:55:39 +0000 (10:55 -0400)]
Add example using the test_check_dyninst and test_installcheck_dyninst.

5 years agotestsuite/systemtap.examples :: add test_{,install}check_dyninst tag to check.ex
William Cohen [Wed, 7 Aug 2019 14:54:00 +0000 (10:54 -0400)]
testsuite/systemtap.examples :: add test_{,install}check_dyninst tag to check.ex
p

5 years agostapbpf pr24811 test :: also test non-constant iteration limit in foreach.stp
Serhei Makarov [Wed, 31 Jul 2019 17:29:48 +0000 (13:29 -0400)]
stapbpf pr24811 test :: also test non-constant iteration limit in foreach.stp

5 years agostapbpf pr23875 bugfix :: allocate actual keysize in foreach to avoid stack clobber
Serhei Makarov [Wed, 31 Jul 2019 17:25:19 +0000 (13:25 -0400)]
stapbpf pr23875 bugfix :: allocate actual keysize in foreach to avoid stack clobber

* bpf-translate.cxx (bpf_unparser::visit_foreach_loop): allocate actual keysize.
* testsuite/systemtap.bpf/bpf_tests/foreach_string.stp: new partial PR23858 testcase,
  added only the parts necessary to trigger a segfault when bugfix not applied.

5 years agorhbz1732514: java runtime requires java-devel
Frank Ch. Eigler [Tue, 30 Jul 2019 14:18:07 +0000 (10:18 -0400)]
rhbz1732514: java runtime requires java-devel

This is not provided automagically via some other dependency,
so we must hard-Require: it ourselves.

5 years agotapset/bpf/conversions.stp :: typo, should be $rc not rc
Serhei Makarov [Tue, 23 Jul 2019 18:35:30 +0000 (14:35 -0400)]
tapset/bpf/conversions.stp :: typo, should be $rc not rc

5 years agoFix aarch64 to properly access arguments for wrapped syscalls
William Cohen [Tue, 23 Jul 2019 18:24:14 +0000 (14:24 -0400)]
Fix aarch64 to properly access arguments for wrapped syscalls

Linux 4.18 added wrappers for aarch64 syscalls that pass a pointer to
a struct pt_regs holding the values for the actual arguments.  The
syscall tapsets initialize CONTEXT->sregs to point at this data
structure.  However, the aarch64 specific register access code was
using the CONTEXT->kregs and just getting the processor register state
when the kprobe triggered rather than the expected arguments in the
data structure being passed into the syscall.  The aarch64 specific
register code now gets the syscall arguments from the correct pt_regs
structure.

5 years agoruntime: adapt to inconsistent export of task_work_add and task_work_cancel
Frank Ch. Eigler [Sat, 20 Jul 2019 17:11:54 +0000 (13:11 -0400)]
runtime: adapt to inconsistent export of task_work_add and task_work_cancel

Ubuntu user <walac> reports their kernel 5.0.0-17-generic #18-Ubuntu
SMP encounters pass-5 errors with an unresolved task_work_cancel()
symbol.  It turns out this is due to a ubuntu-specific
(android-related) kernel commit that exports only task_work_add().
Updated buildrun.cxx and relevant runtime files to separately check
for exportedness of task_work_cancel().

commit 2cc291a463c42641b4146b2507fa4499da35d335
Author: Christian Brauner <christian@brauner.io>
Date:   Wed Jan 16 23:13:25 2019 +0100

    UBUNTU: SAUCE: binder: turn into module

5 years agoEliminate ambiguous python shebangs
William Cohen [Fri, 19 Jul 2019 14:46:59 +0000 (10:46 -0400)]
Eliminate ambiguous python shebangs

Fedora rawhide (31) now throws errors for ambiguous python shebangs.
The systemtap.spec has been modified as outlined in the Fedora feature
wiki page to avoid those ambiguities:

https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error

5 years agoAdd mmwriteback.stp to the list of examples that BPF can run.
William Cohen [Thu, 18 Jul 2019 18:52:47 +0000 (14:52 -0400)]
Add mmwriteback.stp to the list of examples that BPF can run.

5 years agoAdd a test for the BPF raw tracepoints functionality. (PR23866)
William Cohen [Thu, 18 Jul 2019 15:57:50 +0000 (11:57 -0400)]
Add a test for the BPF raw tracepoints functionality. (PR23866)

5 years agoPR23866: Make the bpf backend use BPF raw tracepoints for kernel.trace("*")
William Cohen [Thu, 18 Jul 2019 15:37:37 +0000 (11:37 -0400)]
PR23866: Make the bpf backend use BPF raw tracepoints for kernel.trace("*")

The BPF raw tracepoints provide arguments that better match the
Systemtap lkm kernel tracepoint probes than regular BPF tracepoints.
The BPF backend will use the BPF raw tracepoints unless the user
specifies the old behavior with a --compatible=4.1 option on the
commandline to address.

The new BPF raw tracepoints and their argument are discovered in the
same way as the old BPF tracepoints.  A number of small machine
generated C files are generated with macros are compiled to query the
available tracepoints in the kernel. Debug information describing the
data structures passed into the BPF tracepoints is examined to
determine the type and location of the tracepoint arguments.

Each probe handler BPF code for BPF raw tracepoints is put into a
raw_trace section as the method of registering the BPF raw tracepoints
is different than the regular BPF tracepoints.

The bpf tests have been revised to include the --compatible=4.1 option
for the tests where it makes a difference.

5 years agostapbpf pr24811 bugfix :: don't pop keys.back() on first iteration
Serhei Makarov [Mon, 15 Jul 2019 17:14:25 +0000 (13:14 -0400)]
stapbpf pr24811 bugfix :: don't pop keys.back() on first iteration

If key == NULL, the sorted data for the current foreach loop
has not yet been created. Hence if limit == 0, keys.back() shouldn't
be popped.

* stapbpf/bpfinterp.cxx (map_get_next_key): bugfix for limit == 0.

5 years agostapbpf PR24811 test :: unveil a nasty lurking segfault case in foreach loops
Serhei Makarov [Mon, 15 Jul 2019 16:53:29 +0000 (12:53 -0400)]
stapbpf PR24811 test :: unveil a nasty lurking segfault case in foreach loops

5 years agoAdd some minor tweaks to backtracing add-ons and mention in NEWS
Jafeer Uddin [Thu, 11 Jul 2019 20:20:13 +0000 (16:20 -0400)]
Add some minor tweaks to backtracing add-ons and mention in NEWS

* NEWS: mention new backtracing add-ons
* runtime/regs.h: fix bug with REG_LINK macro for __aarch64__
* tapset/linux/ucontext-unwind.stp: tweak the register-taking backtraces to be more general and work
  with other archs

5 years agoFix errors with new backtrace functions missed in backtracing patch
Jafeer Uddin [Tue, 2 Jul 2019 15:57:07 +0000 (11:57 -0400)]
Fix errors with new backtrace functions missed in backtracing patch

5 years agostapbpf/bpfinterp.cxx pr24758 :: add error info to abort() calls
Serhei Makarov [Tue, 2 Jul 2019 15:26:13 +0000 (11:26 -0400)]
stapbpf/bpfinterp.cxx pr24758 :: add error info to abort() calls

This is still not a proper error reporting scheme but it lets me debug
things in the meantime without having to fire up a debugger just to
understand which place in the code called abort().

* bpfinterp.cxx (stapbpf_abort): New macro, prints reason + aborts.
  (stapbpf_just_abort): New macro, aborts for unspecified reason,
  but still prints the location of the abort() call.
  (stapbpf_stat_get): use stapbpf_abort().
  (bpf_handle_transport_msg): ditto.
  (bpf_interpret): ditto.

5 years agoAdd more capabilities to systemtap backtracing
Jafeer Uddin [Tue, 25 Jun 2019 20:23:41 +0000 (16:23 -0400)]
Add more capabilities to systemtap backtracing

* runtime/sym.c: simplified logic in _stp_snprint_addr because it doesn't need to be that verbose,
  also added ability to have file names and line numbers print in backtraces
* runtime/sym.h: added new symbol flag to generate a more "fuller" backtrace (includes file names
  and line numbers)
* tapset/linux/context-unwind.stp: added new backtrace tapset function that also prints file names
* tapset/linux/ucontext-unwind.stp: added new backtrace tapset function that also prints file names
  and functions to allow backtraces using a user provided pc & sp.

5 years agoMake use of dwarf_aggregate_size to find size attr of types
Jafeer Uddin [Tue, 25 Jun 2019 19:14:26 +0000 (15:14 -0400)]
Make use of dwarf_aggregate_size to find size attr of types

When trying to probe go lang structure pointers, was getting
"cannot find byte_size attribute" errors. This patch fixes
that issue by using the dwarf_aggregate_size function in
elfutils.

5 years agoruntime: stp_tracepoint_module_notifier rc change
Frank Ch. Eigler [Mon, 24 Jun 2019 21:05:53 +0000 (17:05 -0400)]
runtime: stp_tracepoint_module_notifier rc change

Some LKML traffic indicates NOTIFY_OK may be better than NOTIFY_DONE (0)
to return in a routine success case.  Following suit for this site.
OTOH leaving another instance in runtime/linux/symbols.c alone for now.

https://lkml.org/lkml/2019/6/24/880

Reported-By: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
5 years agoPR11353: enable probe elision optimization
Sagar Patel [Fri, 7 Jun 2019 15:36:07 +0000 (11:36 -0400)]
PR11353: enable probe elision optimization

Previously, the compiler only issued warnings for probes with empty
handlers. With the new implementation, the compiler additionally
elides probes with empty handlers. The joining of probes to their
groups has been delayed until after this optimization is performed.

Many tests in the testsuite were built under the assumption that
probes with empty handlers are not elided. Consequently, many of
them stopped working and were fixed.

1) Removed probes with empty handlers from session.
2) Issued warnings whenever a probe is elided.
3) Added tests which check for correct probe elision.
4) Fixed tests affected by probe elision.
5) Updated NEWS.

5 years agostapbpf pr24528 :: new bpf embeddedcode /* userspace */ annotation
Serhei Makarov [Wed, 26 Jun 2019 20:26:20 +0000 (16:26 -0400)]
stapbpf pr24528 :: new bpf embeddedcode /* userspace */ annotation

* bpf-translate.cxx (bpf_unparser::visit_embeddedcode): guard against
  using embeddedcode blocks marked /* userspace */ in in-kernel BPF.

* tapset/bpf/context.stp (_stp_get_target): mark userspace-only.
* tapset/bpf/timestamp_gtod.stp (__gtod_get_ns): mark userspace-only.

5 years agostapbpf PR24543 oops in previous commit (mark_active_cpus)
Serhei Makarov [Mon, 24 Jun 2019 18:42:30 +0000 (14:42 -0400)]
stapbpf PR24543 oops in previous commit (mark_active_cpus)

5 years agoPR24543 just-in-case fix :: stapbpf breaks when cpu0 is disabled
Serhei Makarov [Mon, 24 Jun 2019 18:19:16 +0000 (14:19 -0400)]
PR24543 just-in-case fix :: stapbpf breaks when cpu0 is disabled

Certain perf_events constructs created by stapbpf must be pinned to
one cpu, it doesn't matter which. Previously cpu0 was used.

*Very rarely* it's possible for cpu0 to be disabled.
(It's not recommended,

In that case stapbpf will fail with a super confusing error.

This patch improves readability and allows stapbpf to use a fallback cpu.
cpu0 is still preferred whenever available.

* stapbpf/stapbpf.cxx (default_cpu): New global.
  (mark_active_cpus): Set default_cpu, prefer cpu0 whenever possible.
  (create_group_fds): Use default_cpu, not cpu0.
  (register_uprobes): Ditto.
  (register_kprobes): Ditto.
  (register_tracepoints): Ditto.
  (register_timers): Ditto.
  (register_perf): Ditto.
  (load_bpf_file): Clarifying comment on when active cpu info is checked,
    use default_cpu for uctx bpf_transport_context.

5 years agoPR10280 :: revert prior fix as it does not work for all kernel configs
Serhei Makarov [Fri, 21 Jun 2019 22:02:32 +0000 (18:02 -0400)]
PR10280 :: revert prior fix as it does not work for all kernel configs

Will replace with a better fix only if I find a better solution.

5 years agostapbpf PR24528 housekeeping :: check for userspace-only constructs
Serhei Makarov [Fri, 21 Jun 2019 15:06:08 +0000 (11:06 -0400)]
stapbpf PR24528 housekeeping :: check for userspace-only constructs

And signal an error if they're used in the wrong locations.

* bpf-translate.cxx (bpf_unparser::visit_embeddedcode): comment
  noting future checking (we could mark tapset functions userspace-only).
  (bpf_unparser::visit_for_loop): userspace-only feature.
  (bpf_unparser::visit_foreach_loop): userspace-only feature.
  (bpf_unparser::emit_print_format): sprintf is userspace-only.
  (bpf_unparser::visit_stat_op): stats extraction is userspace-only.

* bpf-internal.h (enum bpf_target): clarify comment.

5 years agostap-prep: adapt to ubuntu 18.04
Frank Ch. Eigler [Fri, 21 Jun 2019 14:39:19 +0000 (10:39 -0400)]
stap-prep: adapt to ubuntu 18.04

The /proc/version_signature file has disappeared there; copying
Debian parse of /proc/version instead.

5 years agostapbpf PR24528 housekeeping :: introduce enum bpf_target
Serhei Makarov [Thu, 20 Jun 2019 17:14:39 +0000 (13:14 -0400)]
stapbpf PR24528 housekeeping :: introduce enum bpf_target

Some housekeeping work to allow generating different code for the
less-restrictive userspace BPF interpreter vs. the more-restrictive
in-kernel BPF JIT interpreter.

* bpf-internal.h (enum bpf_target): New enum to track target BPF version.
  (struct program::target): New field,
  tracks the BPF version this program is intended for.
  (struct program::program): Require target to be specified on creation.
* bpf-base.cxx (struct program::program): Initialize target field.

* bpf-translate.cxx (translate_bpf_pass): Specify targets of each program,
  for now begin/end probes are being emitted for userspace BPF,
  all other probes are being emitted for in-kernel BPF.

Down the line we may have a userspace timer probe (PR23477)..

5 years agoAdd ansi_colors and ansi_colors2 to list of bpf compatible examples.
William Cohen [Tue, 18 Jun 2019 16:44:21 +0000 (12:44 -0400)]
Add ansi_colors and ansi_colors2 to list of bpf compatible examples.

5 years agostapbpf PR23559 :: handle octal and hex escape codes
Serhei Makarov [Tue, 18 Jun 2019 16:28:30 +0000 (12:28 -0400)]
stapbpf PR23559 :: handle octal and hex escape codes

* bpf-translate.cxx (translate_escapes): handle numeric escapes.

5 years agostap hash.cxx PR10280 related fix :: also hash guru mode flag
Serhei Makarov [Mon, 17 Jun 2019 20:05:03 +0000 (16:05 -0400)]
stap hash.cxx PR10280 related fix :: also hash guru mode flag

Since PR10280 introduces the idea that guru-mode modules may be
built differently, we need to cache them separately from
non-guru-mode modules.

* hash.cxx (get_base_hash): Add guru-mode flag to hashed info.

5 years agostapbpf next.stp :: add diagnostic print for PR24659
Serhei Makarov [Mon, 10 Jun 2019 17:17:02 +0000 (13:17 -0400)]
stapbpf next.stp :: add diagnostic print for PR24659

5 years agoPR12025: use the decimal or hex format specifier respectively in
Sagar Patel [Wed, 5 Jun 2019 17:10:05 +0000 (13:10 -0400)]
PR12025: use the decimal or hex format specifier respectively in
the automatic printing of integers and pointers

Previously, the detailed types structures were not propagated to
global variables. Consequently, integers and pointers were both
printed using the hex format specifier. With the new implementation,
pointer and integer types can be differentiated.

1) Propagated detailed type structures to global variables.
2) Applied warning for potential type mismatch.
3) Added tests which check for hex and decimal format specifiers.

5 years agoAdd mmfilepage.stp to the list of examples that work with the bpf backend. wcohen/bpf_target
William Cohen [Fri, 7 Jun 2019 19:44:46 +0000 (15:44 -0400)]
Add mmfilepage.stp to the list of examples that work with the bpf backend.

5 years ago@count() and @sum() should provide 0 for empty entries
William Cohen [Fri, 7 Jun 2019 19:15:27 +0000 (15:15 -0400)]
@count() and @sum() should provide 0 for empty entries

When compiled with the bpf backend mmfilepage.stp would have a
segmentation fault when printing out recorded information in the probe
end.  This was cause by a foreach using an index from one global array
for other global arrays that did not have entries for some of those
indices.  The code computing the @count() and @sum() values did not
handle the situation where there were no statistics for that
particular index in the global array.  The proper default is that
@count() and @sum() in those cases should be 0.

5 years agoAdd support for the target() function and -x option for bpf backend.
William Cohen [Thu, 6 Jun 2019 18:28:47 +0000 (14:28 -0400)]
Add support for the target() function and -x option for bpf backend.

5 years agostapbpf PR24639 :: basic support for next statement
Serhei Makarov [Wed, 5 Jun 2019 20:28:12 +0000 (16:28 -0400)]
stapbpf PR24639 :: basic support for next statement

Only support next statement in probes (terminate probe),
not in functions (relates to not-supported overloading).

* bpf-translate.cxx (bpf_unparser::visit_next_statement): New method.
* testsuite/systemtap.bpf/bpf_tests/next_stp: New testcase.

5 years agoPR24343: fix uninitialized variable warning and empty output of stap -L for return...
Sagar Patel [Wed, 22 May 2019 20:09:19 +0000 (16:09 -0400)]
PR24343: fix uninitialized variable warning and empty output of stap -L for return probes

The uninitialized variable warning was caused by exit syscalls as they don't have
conventional return strings. The incorrect output of stap -L was caused by synthetic
probes created from @entry variables which resulted in miscalculation of the set
intersection check.

1) Set 'never' filters for exit syscalls.
2) Applied synthetic probe flag to filter out synthetic probes from the set intersection.
3) Added tests which check for correct flagging of synethic probes.

5 years agoAdd aarch64 stapdyn support.
Stan Cox [Wed, 29 May 2019 02:53:14 +0000 (22:53 -0400)]
Add aarch64 stapdyn support.

* runtime/dyninst/regs.c (_stp_print_regs): New for aarch64
* runtime/dyninst/stapdyn.h (struct pt_regs): New for aarch64 since
  asm/ptrace.h does not define it.
* runtime/dyninst/uprobes-regs.c (enter_dyninst_uprobe_regs):  Add aarch64.
* stapdyn/mutatee.cxx (get_dwarf_registers):  Add aarch64.

5 years agoAdd a couple more examples that now work with the bpf code generation:
William Cohen [Thu, 23 May 2019 20:41:18 +0000 (16:41 -0400)]
Add a couple more examples that now work with the bpf code generation:

  apps/libguestfs_log.stp
  network/sk_stream_wait_memory.stp

5 years agoProvide gettimeofday_* functions for bpf backend
William Cohen [Thu, 23 May 2019 18:46:34 +0000 (14:46 -0400)]
Provide gettimeofday_* functions for bpf backend

BPF has a helper function ktime_get_ns() that provides a nanosecond
time from the time that the machine was booted.  When working across
machines really want to have a timestamp based on gettimeofday.  This
new BPF backend tapset computes an offset and scaling to convert the
ktime_get_ns() values into gettimeofday_*().  This will allow easier
comparison of timestamped traces between machines.

5 years agoRHBZ1613157: support powerpc64 sdt.h rN,rM operand syntax
Frank Ch. Eigler [Tue, 21 May 2019 23:28:58 +0000 (19:28 -0400)]
RHBZ1613157: support powerpc64 sdt.h rN,rM operand syntax

This syntax appears in some ppc64le binaries, including of stap
itself.  It means the same thing as "0(rN,rM)" on other architectures.

5 years agoconfigury: make python*-config work on rhel6 again
Frank Ch. Eigler [Fri, 17 May 2019 15:32:15 +0000 (11:32 -0400)]
configury: make python*-config work on rhel6 again

commit 5dabffcf0e77d7479ad stopped searching for all hypothetical
variants of "python{2,}-config".  RHEL6 sports python2 binaries
but only a python-config, which was missed by that change.  Let's
return to searching for all these aliases.

Reported-By: <mcermak@redhat.com>
5 years agoMark helloworld as another test that can run as bpf.
William Cohen [Fri, 17 May 2019 14:37:20 +0000 (10:37 -0400)]
Mark helloworld as another test that can run as bpf.

5 years agotestsuite/systemtap.examples :: merge cachestat and cachestat_bpf
Serhei Makarov [Thu, 16 May 2019 21:27:43 +0000 (17:27 -0400)]
testsuite/systemtap.examples :: merge cachestat and cachestat_bpf

5 years agotestsuite/systemtap.examples :: add test_{,install}check_bpf tag to check.exp
Serhei Makarov [Thu, 16 May 2019 21:27:05 +0000 (17:27 -0400)]
testsuite/systemtap.examples :: add test_{,install}check_bpf tag to check.exp

5 years agostapbpf/bpfinterp.cxx (map_get_next_key) :: try to pass all warnings
Serhei Makarov [Wed, 15 May 2019 20:48:34 +0000 (16:48 -0400)]
stapbpf/bpfinterp.cxx (map_get_next_key) :: try to pass all warnings

turns out RHEL7 gcc did not understand __attribute__ ((nonstring)).
This code has extra paranoia in adding a NUL beyond the area
overwritten by strncpy. Switch from strncpy to memcpy since bpf
syscall is treating everything as opaque memory.

5 years agospin-rawhide: usability tweaks
Frank Ch. Eigler [Tue, 14 May 2019 01:44:34 +0000 (21:44 -0400)]
spin-rawhide: usability tweaks

Print & clean up the tmp directory, and tolerate absent changelog changes

5 years agoPR gdb/24541: tweak i386 sys/sdt.h register operand options
Frank Ch. Eigler [Tue, 14 May 2019 01:26:33 +0000 (21:26 -0400)]
PR gdb/24541: tweak i386 sys/sdt.h register operand options

PR gcc/80115 and gdb/24541 had a fight about how values stored in
registers with aliases (e.g. al, ax, rax) should be named.  We all
lost.  :-O

This patch tweaks the i386 controls to bias gcc toward wider aliases,
and adds a comment block explaining the situation.  Unfortunately,
cases still exist where a sys/sdt.h note consumer has to use
arch-specific heuristics to decode gcc's intent.

5 years agostapbpf fix for GCC9 :: use ((nonstring)) attr for map key buffers
Serhei Makarov [Mon, 13 May 2019 18:55:30 +0000 (14:55 -0400)]
stapbpf fix for GCC9 :: use ((nonstring)) attr for map key buffers

Newer gcc9 raises suspicions on copying BPF_MAXSTRINGLEN to
BPF_MAXSTRINGLEN_PLUS sized buffer. Tentative fix, worth reviewing
these issues further.

* stapbpf/bpfinterp.cxx (map_get_next_key): add
  __attribute__ ((nonstring)) to signal how buffers will be used.

5 years agosystemtap-service.8 man page typo
Carlos O'Donell [Mon, 13 May 2019 21:07:07 +0000 (17:07 -0400)]
systemtap-service.8 man page typo

conf.d files are named foo.conf

5 years agostapbpf.cxx: typo fix
Frank Ch. Eigler [Fri, 10 May 2019 12:21:29 +0000 (08:21 -0400)]
stapbpf.cxx: typo fix

5 years agostapbpf/stapbpf.cxx :: fix perf_fds packing code for non-contiguous CPUs
Serhei Makarov [Thu, 9 May 2019 20:47:22 +0000 (16:47 -0400)]
stapbpf/stapbpf.cxx :: fix perf_fds packing code for non-contiguous CPUs

Spotted an error, not the same error that Coverity thinks is happening.
Validity of perf_fds[cpu] (not perf_fds[i]) is indicated by cpus_active[cpu].
There are some additional issues I missed in new code since I first fixed the
code to work with noncontiguous CPUs.

* stapbpf/stapbpf.cxx (perf_event_loop): assign perf_fds[cpu], not perf_fds[i] !!,
maintain i -> cpu mapping to retrieve correct perf_header and transport_context.

5 years agostapbpf/stapbpf.cxx :: placate the gods of Coverity
Serhei Makarov [Thu, 9 May 2019 20:45:11 +0000 (16:45 -0400)]
stapbpf/stapbpf.cxx :: placate the gods of Coverity

* stapbpf/stapbpf.cxx (instantiate_maps): UNUSED_VALUE #if0 out unused code;
NEGATIVE_RETURNS handle error return from sysconf(_SC_NPROCESSORS_CONF),
previous sysconf fix was valid but it might make sense to print a warning.
(register_tracepoints): RESOURCE_LEAK close fd on read failure.

5 years agostapbpf/bpfinterp.cxx :: placate the gods of Coverity
Serhei Makarov [Thu, 9 May 2019 20:43:36 +0000 (16:43 -0400)]
stapbpf/bpfinterp.cxx :: placate the gods of Coverity

* bpf-internal.h (BPF_MAXSTRINGLEN_PLUS): new define, BPF_MAXSTRINGLEN+1.
* stapbpf/bpfinterp.cxx (map_get_next_key): BUFFER_SIZE_WARNING use bigger buffer.
(bpf_interpret): UNUSED_VALUE memset regs to 0x0,
OVERFLOW_BEFORE_WIDEN indicate in 32-bit LSH operation that widening is inappropriate.

5 years agoPR23074: fix guru mode issue with generated calls to register get/set
Jafeer Uddin [Thu, 9 May 2019 20:18:46 +0000 (16:18 -0400)]
PR23074: fix guru mode issue with generated calls to register get/set

5 years agostapbpf.cxx sysconf() coverity nit
Frank Ch. Eigler [Thu, 9 May 2019 13:22:08 +0000 (09:22 -0400)]
stapbpf.cxx sysconf() coverity nit

5 years agoupdate-docs pass
Frank Ch. Eigler [Tue, 7 May 2019 22:35:02 +0000 (18:35 -0400)]
update-docs pass

5 years agobegin next release work: bump version to Ultimate Answer
Serhei Makarov [Tue, 7 May 2019 21:21:39 +0000 (17:21 -0400)]
begin next release work: bump version to Ultimate Answer

5 years agosystemtap.spec: inconsequential(?) typo
Serhei Makarov [Tue, 7 May 2019 18:59:39 +0000 (14:59 -0400)]
systemtap.spec: inconsequential(?) typo

5 years agoNEWS, systemtap.spec :: bump release by 1d release-4.1
Serhei Makarov [Tue, 7 May 2019 14:36:41 +0000 (10:36 -0400)]
NEWS, systemtap.spec :: bump release by 1d

5 years agoNEWS :: 4.1 release is go
Serhei Makarov [Mon, 6 May 2019 18:21:12 +0000 (14:21 -0400)]
NEWS :: 4.1 release is go

5 years agoruntime: on kernels >= 4.0, multiple concurrent systemtap scripts
Frank Ch. Eigler [Mon, 6 May 2019 22:27:09 +0000 (22:27 +0000)]
runtime: on kernels >= 4.0, multiple concurrent systemtap scripts

This old kernel commit:

commit 190afd81e4a5f24d1db6c774ca7f18acde2fca30
Author: Al Viro <viro@zeniv.linux.org.uk>
Date:   Sun Jan 25 13:55:55 2015 -0500

    debugfs: split the beginning and the end of __create_file() off

appears to have encoded duplicate $debugfs/systemtap directory
existence into an -EEXIST error code, which we didn't handle.
We now treat it as though it were NULL.  Tested on 5.1-rc7.

5 years agoexamples index regen
Frank Ch. Eigler [Mon, 6 May 2019 16:38:40 +0000 (12:38 -0400)]
examples index regen

5 years agoprerelease docs regen
Frank Ch. Eigler [Mon, 6 May 2019 16:35:58 +0000 (12:35 -0400)]
prerelease docs regen

5 years agoPRERELEASE content bump
Frank Ch. Eigler [Mon, 6 May 2019 16:25:09 +0000 (12:25 -0400)]
PRERELEASE content bump

5 years agocopyright year updates
Frank Ch. Eigler [Mon, 6 May 2019 16:18:06 +0000 (12:18 -0400)]
copyright year updates

5 years agoNEWS :: mention ktime_get_real_fast_ns() from commit e2e13220fb
Serhei Makarov [Mon, 6 May 2019 15:48:54 +0000 (11:48 -0400)]
NEWS :: mention ktime_get_real_fast_ns() from commit e2e13220fb

5 years agoNEWS :: mention BPF tapset functions
Serhei Makarov [Sat, 4 May 2019 20:41:07 +0000 (16:41 -0400)]
NEWS :: mention BPF tapset functions

5 years agoadd recent AUTHORS
Serhei Makarov [Sat, 4 May 2019 17:14:00 +0000 (13:14 -0400)]
add recent AUTHORS

5 years agoNEWS: document added support for unhandled DW_OP_GNU_parameter_ref
Jafeer Uddin [Fri, 3 May 2019 19:35:36 +0000 (15:35 -0400)]
NEWS: document added support for unhandled DW_OP_GNU_parameter_ref

5 years agostapbpf/stapbpf.8 :: update based on recent stapbpf work
Serhei Makarov [Fri, 3 May 2019 19:00:44 +0000 (15:00 -0400)]
stapbpf/stapbpf.8 :: update based on recent stapbpf work

5 years agoNEWS :: document recent stapbpf additions
Serhei Makarov [Fri, 3 May 2019 18:52:56 +0000 (14:52 -0400)]
NEWS :: document recent stapbpf additions

5 years agoNEWS :: fxi tyops
Serhei Makarov [Fri, 3 May 2019 18:46:03 +0000 (14:46 -0400)]
NEWS :: fxi tyops

5 years agoAllow preprocessor.exp test to work with 5.x kernels
William Cohen [Fri, 3 May 2019 18:44:31 +0000 (14:44 -0400)]
Allow preprocessor.exp test to work with 5.x kernels

5 years agostapbpf fixup for 4.1 :: typo in error message
Serhei Makarov [Fri, 3 May 2019 15:54:35 +0000 (11:54 -0400)]
stapbpf fixup for 4.1 :: typo in error message

5 years agostapbpf PR23476 fixup :: typo in stat2.stp testcase
Serhei Makarov [Fri, 3 May 2019 15:42:19 +0000 (11:42 -0400)]
stapbpf PR23476 fixup :: typo in stat2.stp testcase

This page took 0.067545 seconds and 5 git commands to generate.