]> sourceware.org Git - systemtap.git/log
systemtap.git
6 years agoPR23359: impose security constraints on @kderef, @kregister
Jafeer Uddin [Tue, 3 Jul 2018 16:12:09 +0000 (12:12 -0400)]
PR23359: impose security constraints on @kderef, @kregister

* parse.cxx: add privilege check for @kderef and @kregister
* testsuite/parseko/at_kderef.stp: New file to test privilege check
* testsuite/parseko/at_kregister.stp: New file to test privilege check
* testsuite/parseok/at_kderef.stp: New file to test privilege check
* testsuite/parseok/at_kregister.stp: New file to test privilege check

6 years agoPR23356: stap-serverd: switch back to dbm: nss databases
Frank Ch. Eigler [Sat, 30 Jun 2018 21:00:20 +0000 (17:00 -0400)]
PR23356: stap-serverd: switch back to dbm: nss databases

The sqlite sql: nss ones seem to have different
authentication/login/password protocols as on rawhide, and our logic
can't quite play right with them.  Maybe it's due to possible
concurrent access from a stap-serverd and stap client (when both run
as root).  Whatever it is, plain dbm: seems to restore function,
even if with a warning about "legacy database".

6 years agoPR23356: nsscommon: adopt new defaults/requirements of rawhide nss
Frank Ch. Eigler [Sat, 30 Jun 2018 03:04:48 +0000 (23:04 -0400)]
PR23356: nsscommon: adopt new defaults/requirements of rawhide nss

The previous key size of 1024 has become invalid on rawhide, where
2048 is the new minimum.  Use 4096 in generate_private_key().  Also,
switch add_cert_db_prefix() to the sql: cert-db prefix, in order to
nuke the -8015 SEC_ERROR_LEGACY_DATABASE error.

6 years agoPR23356: stap-serverd: fix private key disposal race condition
Frank Ch. Eigler [Sat, 30 Jun 2018 03:01:33 +0000 (23:01 -0400)]
PR23356: stap-serverd: fix private key disposal race condition

Previous code made it possible for an incoming request to be processed
in a handle_connection() thread at the same time as the privkey to it
would be disposed-of in the accept_connections() loop.  (This could
happen if the latter encountered a cert-validation error immediately
after the handle_connection thread started, thus its loop was exited.)

We now pass a SECKEY_CopyPrivateKey(privKey) to the thread, which it
will dispose of itself.  Thus the main thread can do its prior thing.

6 years agoPR23356: testsuite: relay stap-serverd diagnostic output into systemtap.log
Frank Ch. Eigler [Sat, 30 Jun 2018 03:00:31 +0000 (23:00 -0400)]
PR23356: testsuite: relay stap-serverd diagnostic output into systemtap.log

Previously, this log data was disposed-of.

6 years agoNEWS: minor @deref @register blurb tweak to give example
Frank Ch. Eigler [Sat, 30 Jun 2018 02:59:54 +0000 (22:59 -0400)]
NEWS: minor @deref @register blurb tweak to give example

6 years agoPR23356: stap-serverd: close client socket upon error
Frank Ch. Eigler [Fri, 29 Jun 2018 23:58:49 +0000 (19:58 -0400)]
PR23356: stap-serverd: close client socket upon error

setupSSLSocket() was designed to return NULL on error, but it didn't
clean up its partially constructed / given sockets at all.  They'd
stay open, and a remote client would be left to wait indefinitely.

6 years agoPR23356: improve stap-serverd nss diagnostics
Frank Ch. Eigler [Fri, 29 Jun 2018 23:25:45 +0000 (19:25 -0400)]
PR23356: improve stap-serverd nss diagnostics

On rawhide, we can get certificate verification errors that previous
control flow kept quiet about.  When it did talk about them, the error
codes were out of nssError()'s assumed range.  Now nssError() tries
the PR_ErrorTo* functions regardless of error number range.

6 years agoPR22310: build parser syntax for all the new staptree type
Jafeer Uddin [Fri, 29 Jun 2018 14:33:39 +0000 (10:33 -0400)]
PR22310: build parser syntax for all the new staptree type

* parse.cxx: add parser support for new target_register and target_deref types

* testsuite/systemtap.base/*: new files to test the new types

* man/stap.1.in: add information on new types

* NEWS: mention new types

6 years agoUpdate fedora_install_package.py to handle Centos and Fedora packages.
David Smith [Thu, 28 Jun 2018 19:26:51 +0000 (14:26 -0500)]
Update fedora_install_package.py to handle Centos and Fedora packages.

* httpd/docker/fedora_install_package.py (split_nvra): New function.
  (build_id_symlink_is_valid): Ditto.
  (PkgSystem.build_id_is_valid): If we don't have a build id, pretend it
  matched. Handle Fedora vs. Centos build id differences.
  (PkgSystem.pkg_install): Don't run scripts when installing rpms. They
  can cause issues. Handle the case where 'dnf debuginfo' installs the
  wrong version of the debuginfo rpm.

6 years agotapset: add string quoting and escape special characters in prometheus.stpm
Aaron Merey [Thu, 28 Jun 2018 18:20:22 +0000 (14:20 -0400)]
tapset: add string quoting and escape special characters in prometheus.stpm

6 years agoSimplify the initialization logic for cpu_throttle.stp
William Cohen [Wed, 27 Jun 2018 14:56:44 +0000 (10:56 -0400)]
Simplify the initialization logic for cpu_throttle.stp

Rather than ensuring there is an instance for each of the cpus on the
machine just make sure there is an instance for cpu0 to avoid having
an empty array. As processors are throttled the other instances will
be added.

6 years agoAdd the cpu_throttle.stp example to stap-exporter scripts
William Cohen [Wed, 27 Jun 2018 03:16:20 +0000 (23:16 -0400)]
Add the cpu_throttle.stp example to stap-exporter scripts

The cpu_throttle.stp script monitors Intel x86 cpu throttling due to
power and thermal constraints.  The output is formatted to be
consumable by prometheus.

6 years agoPR23160,PR14690: support /* guru */ embedded-C for CONTEXT->sregs
Frank Ch. Eigler [Thu, 28 Jun 2018 02:32:34 +0000 (22:32 -0400)]
PR23160,PR14690: support /* guru */ embedded-C for CONTEXT->sregs

For the newfangled syscall support, an embedded-C function is needed
to set the CONTEXT->sregs field from any of the new gajillion syscall
probe aliases.  Since we want that ->sregs field trustworthy, and not
callable by the user with some random junk pointer, we want this
function to be only callable from the tapset, but not replicated (so
not "private" in each file).

To do this, we extend the parser/elaborator to extend the logic of /*
guru */ markup in embedded-C functions.  Namely, calls to such
functions from the tapset (which is parsed with privileged flags) are
now permitted, kind of as if they were a private copy in each tapset
file that called them.  The stapfile privileged field, which has
fallen into disuse, is brought back to life and propagates the
parse-time pf_guru flag.  It's used with a new visitor that checks
each caller-callee relationship in function calls, moving some similar
code from staptree.cxx.

The syscall.read aliases are adapted to use the new single copy of
__set_syscall_pt_regs() in aux2_syscalls.stp.

6 years agotapset: add stap-exporter utility macros and probe alias
Aaron Merey [Wed, 27 Jun 2018 20:51:57 +0000 (16:51 -0400)]
tapset: add stap-exporter utility macros and probe alias

* prometheus.stp: defines a "prometheus" probe as an alias for a procfs read probe

* prometheus.stpm: defines utility macros that create metrics from arrays and writes
  them to a procfs file (intended for use within prometheus probes).

6 years agorhbz1595178: add some inter-subrpm cross-version Conflicts:
Frank Ch. Eigler [Wed, 27 Jun 2018 01:16:27 +0000 (21:16 -0400)]
rhbz1595178: add some inter-subrpm cross-version Conflicts:

Some files (e.g. some man pages, some localization) shared between
subrpms make it impossible to upgrade only some subrpms across
versions, due to file conflicts.  We now add some Conflicts: so as to
prevent concurrent installation of different versions of some subrpms.

6 years agobpf testsuite :: CHECK_ONLY facility for bpf.exp
Serhei Makarov [Tue, 26 Jun 2018 17:51:03 +0000 (13:51 -0400)]
bpf testsuite :: CHECK_ONLY facility for bpf.exp

6 years agosystemtap.spec: pass %{with_bpf} to %configure
Frank Ch. Eigler [Tue, 26 Jun 2018 16:18:00 +0000 (12:18 -0400)]
systemtap.spec: pass %{with_bpf} to %configure

6 years agoconfigury: add --without-bpf configure option
Frank Ch. Eigler [Tue, 26 Jun 2018 16:16:04 +0000 (12:16 -0400)]
configury: add --without-bpf configure option

Fully autoconf'ing bpf capabilities is error-prone, so provide a blunt
instrument.

6 years agobpf testsuite :: basic printf.stp requirements for log() (commented out for now)
Serhei Makarov [Tue, 26 Jun 2018 17:11:49 +0000 (13:11 -0400)]
bpf testsuite :: basic printf.stp requirements for log() (commented out for now)

6 years agobpf-translate.cxx :: slightly clearer diagnostic for 'unhandled statement'
Serhei Makarov [Tue, 26 Jun 2018 15:48:19 +0000 (11:48 -0400)]
bpf-translate.cxx :: slightly clearer diagnostic for 'unhandled statement'

6 years agoPR23160,PR14690: 32-on-64 bit fixes
Frank Ch. Eigler [Mon, 25 Jun 2018 16:34:43 +0000 (12:34 -0400)]
PR23160,PR14690: 32-on-64 bit fixes

After reported crashes with the syscalls.* test cases, found that
32-on-64 bits were b0rked, because pt_regs* addresses were being
truncated and yet later dereferenced in kernel space.

To simplify analysis, added a pt_regs *sregs to the common probe
context, which signifies 'syscall mode' register dumps.  This is
different from normal kregs (kernel-space, normal abi, 64-bit-only)
and uregs (user-space, normal abi, either 32- or 64-bit), and needs
custom processing in _stp_syscall_nr and especially _stp_arg2.

The sregs-setter embedded-C function __set_syscall_pt_regs(r)
needs to be private/tapset-guru, but we lack proper /* markup */
for that particular mode.  So that function currently needs to
be replicated as private inside each sysc_* file, ugh.  Not
for long though.

After this patch, while this doesn't quite pass, but the read
parts look good:

sudo make installcheck RUNTESTFLAGS=nd_syscall.exp\ syscall.exp CHECK_ONLY="readwrite"

6 years agoRestore the old mode for exporter.py
William Cohen [Fri, 22 Jun 2018 02:25:41 +0000 (22:25 -0400)]
Restore the old mode for exporter.py

6 years agoPR22193 - delete regex_noelide.stp as the proper semantics are currently undecided
Serhei Makarov [Thu, 21 Jun 2018 21:24:48 +0000 (17:24 -0400)]
PR22193 - delete regex_noelide.stp as the proper semantics are currently undecided

6 years agoClean up the exporter.py formatting with autopep8
William Cohen [Thu, 21 Jun 2018 18:06:26 +0000 (14:06 -0400)]
Clean up the exporter.py formatting with autopep8

The formatting in exporter.py did not follow the PEP8 style guide and
pylint would complain about a lot of minor formatting issues.  Ran
exporter.py through autopep8 to eliminate those warnings.

6 years agobpf translator: fix comment typo
Aaron Merey [Wed, 20 Jun 2018 22:47:44 +0000 (18:47 -0400)]
bpf translator: fix comment typo

6 years agotestsuite: fix formatting of bpf test increment2.stp
Aaron Merey [Wed, 20 Jun 2018 22:42:53 +0000 (18:42 -0400)]
testsuite: fix formatting of bpf test increment2.stp

6 years agobpf translator: binary operations now act on copy of left operand.
Aaron Merey [Wed, 20 Jun 2018 22:26:36 +0000 (18:26 -0400)]
bpf translator: binary operations now act on copy of left operand.

Needed for cases where evaluating right operand causes side effects that
mutate the left (ex. x + x++).

6 years agoUse equivalent non-dwarf probe points for example scripts
William Cohen [Wed, 20 Jun 2018 20:35:03 +0000 (16:35 -0400)]
Use equivalent non-dwarf probe points for example scripts

The dwarf-based kernel.function probes require the installation of
kernel-debuginfo.  The examples can be just as easily implemented with
the non-dwarf kprobe.function probes and eliminate the need to install
kernel-debuginfo.

6 years agoignore coverity / rpmmacro
Frank Ch. Eigler [Wed, 13 Jun 2018 20:11:35 +0000 (16:11 -0400)]
ignore coverity / rpmmacro

6 years agotapset: introduce tapset/linux/aux2_syscalls.stp for leaf embedded-c functions
Frank Ch. Eigler [Wed, 20 Jun 2018 18:00:15 +0000 (14:00 -0400)]
tapset: introduce tapset/linux/aux2_syscalls.stp for leaf embedded-c functions

The _stp_syscall_nr embedded-C function may well be needed from the
tracepoint-flavoured syscall probe aliases.  For this, it would be
undesirable to necessarily drag in all of the material in
aux_syscalls.stp.  So split out a new little aux2_syscall.stp with
just the basics that minimal nd2_syscall or tp_syscall jobs might
need.

6 years agoPR23160,PR14690: uregs setter macro for use from kernel context
Frank Ch. Eigler [Mon, 18 Jun 2018 18:57:21 +0000 (14:57 -0400)]
PR23160,PR14690: uregs setter macro for use from kernel context

When a syscall interjection mechanism gives us a pt_regs*
structure for the syscall parameters/context, we can pretend
as though it were a user-space probe.

function __set_usermode_pt_regs(r)

6 years agoPR23160,PR14690: adapt syscall.read for 4.17 __ARCH_sys_FOO and sys_enter/exit
Frank Ch. Eigler [Mon, 18 Jun 2018 18:51:18 +0000 (14:51 -0400)]
PR23160,PR14690: adapt syscall.read for 4.17 __ARCH_sys_FOO and sys_enter/exit

Sketch for the syscall.read and .return probe aliases only.

6 years agodrop unused recursion_guard code from elaborate.cxx
Frank Ch. Eigler [Sun, 17 Jun 2018 13:46:41 +0000 (09:46 -0400)]
drop unused recursion_guard code from elaborate.cxx

6 years agopass2 elaboration: tweak diagnostics
Frank Ch. Eigler [Sat, 16 Jun 2018 08:36:00 +0000 (04:36 -0400)]
pass2 elaboration: tweak diagnostics

While debugging 4.17-style syscalls, it was tricky to figure out the
probe derivation process while it was underway, with nested aliases
and optional/sufficient probe points.  Tweak verbosity numbers and
messages to give a good overview at the --vp 02 level.

While in the vicinity, introduced a session
suppress_costly_diagnostics counter, which is used to suppress
levenshtein suggestions for optional/sufficient probe points.
These probe points are expected to fail, and no messages will
be printed for them anyway, so the levenshtein stuff was a pure
waste.  stap -p2 run time time for scripts like

    probe kernel.function("foo")?,kernel.function("foo")?,
          kernel.function("foo")?,kernel.function("foo")?,
          kernel.function("foo")?,kernel.function("foo")?,
          kernel.function("foo")?,
          never{}'

went from 5.6s to 3.4s on the author's workstation.  The gains
should be noticeable on normal scripts too that use the
syscall tapset.

6 years agostap-exporter: add install script
Aaron Merey [Fri, 15 Jun 2018 21:17:27 +0000 (17:17 -0400)]
stap-exporter: add install script

* Install: installs the exporter as a systemd service

* exporter.py: path to stap binary is no longer hardcoded. All responses
  sent by the server are now plain text.

6 years agoNo longer run the http server as 'root'.
David Smith [Thu, 14 Jun 2018 18:18:24 +0000 (13:18 -0500)]
No longer run the http server as 'root'.

* httpd/backends.cxx (container_backend::generate_module): Handle python
  versions correctly. Add "sudo" to all "buildah" command lines.
* httpd/main.cxx (main): Make sure we're not running as root.
* testsuite/lib/systemtap.exp (systemtap_check_users): Check for the
  'stap-server' user.
* testsuite/lib/http_server.exp: Start the http server as the
  'stap-server' user, not root.
* configure.ac: Add defines to determine if python 2 and python 3 exist on
  the system.
* httpd/Makefile.am: Installs sudoers rule file.
* configure: Regenerated.
* config.in: Ditto.
* httpd/Makefile.in: Ditto.
* util.cxx (get_distro_info): Fixed bug where the version number and
  release number got combined.
* httpd/stap-server.sudoers: New file.

6 years agoImprove the foreach_limit(2).exp test results.
Martin Cermak [Thu, 14 Jun 2018 12:35:26 +0000 (14:35 +0200)]
Improve the foreach_limit(2).exp test results.

Without this update, one can observe following issue with rhel7
powerpc kernels:

=======
 # stap -p4 testsuite/systemtap.maps/foreach_limit.stp
 ...
 /usr/local/share/systemtap/runtime/map.c:275:26: error: ‘a’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
    struct mlist_head *c, *a, *last, *tmp;
                           ^
 /usr/local/share/systemtap/runtime/map.c:275:26: error: ‘a’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
 cc1: all warnings being treated as errors
=======

This problem turns out to start happening after the powerpc kernel
build system started using -O3 instead of -O2 as one can see in
http://vault.centos.org/7.5.1804/os/Source/SPackages/kernel-3.10.0-862.el7.src.rpm

=======
 # powerpc is compiled with -O3, via specfile rpmbuild -- see rhbz1051067.
 # we need to keep consistency here, however, for out of tree kmod builds --
 # see rhbz1431029 for reference
 ifeq ($(SRCARCH), powerpc)
 KBUILD_CFLAGS   += -O3
 else
 KBUILD_CFLAGS   += -O2
 endif
=======

Reverting this change (using -O2 instead of -O3) works the problem around
as well as this systemtap-side update.  For more details, see rhbz1591267.

6 years agostap-exporter: improve configurability, enable automatic session timeouts
Aaron Merey [Wed, 13 Jun 2018 22:50:16 +0000 (18:50 -0400)]
stap-exporter: improve configurability, enable automatic session timeouts

* exporter.conf: lets user specify which scripts the server can run and
  options such as automatic session timeouts and whether sessions are
  launched at startup.

* exporter.py: parse config file, periodically check for sessions that
  need to be terminated due to timeout.

6 years agostap-exporter: initial commit
amerey [Fri, 8 Jun 2018 18:44:23 +0000 (14:44 -0400)]
stap-exporter: initial commit

6 years agorhbz1547238: adapt vfs.add_to_page_cache probes
Paulo Andrade [Tue, 12 Jun 2018 22:48:26 +0000 (18:48 -0400)]
rhbz1547238: adapt vfs.add_to_page_cache probes

The add_to_page_cache_lru variant should also be probed, along with
kernel.function("add_to_page_cache_locked"), but if present, not the
add_to_page_cache variant.  This backward compatibility needed a
little bit of tapset probe point operator jiujitsu to go beyond
Paulo's initial patch.

6 years agoNEWS for 'add name of main script file to dmesg output'
Serhei Makarov [Tue, 12 Jun 2018 19:29:00 +0000 (15:29 -0400)]
NEWS for 'add name of main script file to dmesg output'

6 years agoadd name of main script file to dmesg output
Serhei Makarov [Tue, 12 Jun 2018 19:18:37 +0000 (15:18 -0400)]
add name of main script file to dmesg output

6 years agorefactor string escaping -- move it from stapregex to util.cxx
Serhei Makarov [Tue, 12 Jun 2018 19:17:36 +0000 (15:17 -0400)]
refactor string escaping -- move it from stapregex to util.cxx

6 years agorefactor string escaping -- typo fix
Serhei Makarov [Tue, 12 Jun 2018 18:52:36 +0000 (14:52 -0400)]
refactor string escaping -- typo fix

6 years agoFix (and centralize) temporary directory creation in the http server.
David Smith [Tue, 12 Jun 2018 19:19:01 +0000 (14:19 -0500)]
Fix (and centralize) temporary directory creation in the http server.

* httpd/api.cxx (build_collection_rh::POST): Always ensure there is a
  server temporary directory (otherwise files are attempted to be written
  to the root directory). Use make_temp_directory() to create the client
  temporary directory.
* httpd/utils.cxx (make_temp_dir): New function.
* httpd/utils.h: Added make_temp_dir() prototype.
* httpd/server.cxx (connection_info::postdataiterator): Use
  make_temp_directory().

6 years agocoverity tweaks: confirm a few dynamic_cast<> returns
Frank Ch. Eigler [Sun, 10 Jun 2018 22:51:27 +0000 (18:51 -0400)]
coverity tweaks: confirm a few dynamic_cast<> returns

6 years agoRemove old code from httpd/docker/stap_build_docker_file.py.
David Smith [Mon, 11 Jun 2018 20:27:00 +0000 (15:27 -0500)]
Remove old code from httpd/docker/stap_build_docker_file.py.

* httpd/docker/stap_build_docker_file.py (main): Remove unused code that
  called "docker build".

6 years agoSimplify temporary directory creation in the http container backend.
David Smith [Mon, 11 Jun 2018 16:13:11 +0000 (11:13 -0500)]
Simplify temporary directory creation in the http container backend.

* httpd/backends.cxx (container_backend::generate_module): Instead of
  creating a new temporary directory for the docker file, just create a
  subdirectory of the server temporary directory.

6 years agoversion bump to 4.0 in anticipation of next release
Frank Ch. Eigler [Sat, 9 Jun 2018 13:49:52 +0000 (09:49 -0400)]
version bump to 4.0 in anticipation of next release

6 years agoupdate-po bump release-3.3
Frank Ch. Eigler [Fri, 8 Jun 2018 22:28:59 +0000 (18:28 -0400)]
update-po bump

6 years agorebuild example indexes
Frank Ch. Eigler [Fri, 8 Jun 2018 20:20:23 +0000 (16:20 -0400)]
rebuild example indexes

6 years agomain.cxx: tweak --example error message
Frank Ch. Eigler [Fri, 8 Jun 2018 16:39:43 +0000 (12:39 -0400)]
main.cxx: tweak --example error message

Also print the name of the directory with samples.

6 years agotestsuite: switch smoketest target
Frank Ch. Eigler [Fri, 8 Jun 2018 02:08:35 +0000 (22:08 -0400)]
testsuite: switch smoketest target

With the syscalls being out of action on 4.17, switch to ancient
kernel.function("do_exit") as the target of "make *check" smoke
testing.

6 years agoPRERELEASE nudge further, testing on 4.18-rc
Frank Ch. Eigler [Thu, 7 Jun 2018 21:03:55 +0000 (17:03 -0400)]
PRERELEASE nudge further, testing on 4.18-rc

6 years agocollect delinquent NEWS entries
Frank Ch. Eigler [Thu, 7 Jun 2018 21:03:27 +0000 (17:03 -0400)]
collect delinquent NEWS entries

6 years agoDocumentation update
Jafeer Uddin [Thu, 7 Jun 2018 20:14:59 +0000 (16:14 -0400)]
Documentation update

6 years agoImplement caching for the http container backend.
David Smith [Thu, 7 Jun 2018 19:51:55 +0000 (14:51 -0500)]
Implement caching for the http container backend.

* httpd/backends.cxx (class container_image_cache): New class.
  (container_backend::generate_module): Implement a caching scheme for
  "buildah". If the hash for a docker file matches an existing image,
  reuse the image.

6 years agoPRERELEASE handling
Frank Ch. Eigler [Thu, 7 Jun 2018 19:32:32 +0000 (15:32 -0400)]
PRERELEASE handling

6 years agoAUTHORS bump
Frank Ch. Eigler [Thu, 7 Jun 2018 18:42:54 +0000 (14:42 -0400)]
AUTHORS bump

6 years agoupdate po
Frank Ch. Eigler [Thu, 7 Jun 2018 18:22:01 +0000 (14:22 -0400)]
update po

6 years agoCompat arch support detection improvement for 32-bit only systems.
Martin Cermak [Thu, 7 Jun 2018 18:13:27 +0000 (20:13 +0200)]
Compat arch support detection improvement for 32-bit only systems.

Commit d24c50ecfcf49e4579f475ac52bafb31850cf523 doesn't handle
well 32-bit only systems such as i386.  This update fixes it.

6 years agoPR23226: Added ability to run sample scripts with typing whole path
Jafeer Uddin [Thu, 7 Jun 2018 17:41:11 +0000 (13:41 -0400)]
PR23226: Added ability to run sample scripts with typing whole path

* cmdline.h: Added new command line option '--example'
* cmdline.cxx: Added new command line option '--example'
* session.h: Added new flag 'run_example' to track example option
* session.cxx: Updated constructors to initialize new flag and added
  logic to set run_example flag in parse_cmdline()
* main.cxx: If '--example' is specified then search for script within
  example directory and run it if one hit is found
* man/stap.1.in: Added new entry for '--example' option
* testsuite/buildko/example.stp: New file to test new option
* testsuite/buildok/example.stp: New file to test new option

6 years agoCompat arch support detection for the testsuite.
Martin Cermak [Thu, 7 Jun 2018 14:59:56 +0000 (16:59 +0200)]
Compat arch support detection for the testsuite.

Previously, the compat arch support (e.g. -m32 on a 64-bit system)
was hardcoded per architecture in testsuite/lib/compile_flags.exp.

As the count of rhel products with various compat arch support matrix
grows, it turns out to be more flexible to detect this at run time.

6 years agotestsuite/runok/regex_noelide: innocuous comment edit
Serhei Makarov [Wed, 6 Jun 2018 19:17:39 +0000 (15:17 -0400)]
testsuite/runok/regex_noelide: innocuous comment edit

6 years agohttpd server build fix via <unistd.h> include
Aryeh Weinreb [Wed, 6 Jun 2018 15:46:04 +0000 (11:46 -0400)]
httpd server build fix via <unistd.h> include

6 years agobz1584705: move new macros.systemtap file into -sdt-devel subrpm
Frank Ch. Eigler [Thu, 31 May 2018 17:00:34 +0000 (13:00 -0400)]
bz1584705: move new macros.systemtap file into -sdt-devel subrpm

6 years agoAdd the docker file hash to the container image name.
David Smith [Tue, 5 Jun 2018 16:32:42 +0000 (11:32 -0500)]
Add the docker file hash to the container image name.

* httpd/backends.cxx (container_backend::generate_module): Add the docker
  file hash to the container image name.
* httpd/utils.cxx (get_file_hash): New function.
* httpd/utils.h: Add get_file_hash() prototype.
* httpd/Makefile.am: Add ../mdfour.c to the list of sources.
* httpd/Makefile.in: Regenerated.

6 years agoStarted switching the http container backend to use 'buildah'.
David Smith [Fri, 1 Jun 2018 20:09:23 +0000 (15:09 -0500)]
Started switching the http container backend to use 'buildah'.

* httpd/backends.cxx (container_backend): Initial steps of switching from
  'docker' to 'buildah'.
* httpd/backends.h (backend_base): Updated prototype.
* httpd/api.cxx (build_info): Remove the 'tmp_dir' member.
  (build_info::~build_info): No longer remove the temporary directory.
  (response build_collection_rh::POST): Create the 'client_dir' temporary
  directory.
* httpd/api.h (client_request_data): Rename the 'base_dir' member to
  'server_dir' and add a 'client_dir' member.
  (build_info::module_build): Unzip the 'client.zip' file in the client
  directory, not the server directory. Be sure to look for the module in
  the client directory.
  (client_request_data::~client_request_data): Remove both the client
  directory and the server directory.
  (client_request_data::get_json_object): Update output.
* httpd/docker/centos.json: Tweaked to not run "yum/dnf clean all" after
  the first RUN command.
* httpd/docker/fedora.json: Ditto.

6 years agoMake the RPM macro for tapset dir work on older RHELs too.
Martin Cermak [Fri, 1 Jun 2018 13:08:59 +0000 (15:08 +0200)]
Make the RPM macro for tapset dir work on older RHELs too.

Update commit e914597c18 in a sense of
https://bugzilla.redhat.com/show_bug.cgi?id=888262#c5 .

6 years agoChange the http container backend to do all the image building itself.
David Smith [Thu, 31 May 2018 15:24:04 +0000 (10:24 -0500)]
Change the http container backend to do all the image building itself.

* httpd/backends.cxx (container_backend::generate_module): Just use the
  python script to build the docker file, then build the docker image.
* httpd/docker/stap_build_docker_file.py: Renamed from
  'stap_build_docker_image.py' and only builds the docker file, not the
  docker image.
* httpd/docker/Makefile.am: Rename 'stap_build_docker_image.py' to
  'stap_build_docker_file.py'.
* httpd/docker/Makefile.in: Regenerated.

6 years agoFix a bug where the http server wasn't handling script files properly.
David Smith [Thu, 31 May 2018 14:37:10 +0000 (09:37 -0500)]
Fix a bug where the http server wasn't handling script files properly.

* httpd/backends.cxx (container_backend::generate_module): Fix a
  regression where stap run in the container couldn't find script files.

6 years agoRename the http docker backend to the http container backend.
David Smith [Wed, 30 May 2018 16:32:59 +0000 (11:32 -0500)]
Rename the http docker backend to the http container backend.

* httpd/backends.cxx: No real code change, but in preparation for the
  switch from using "docker" to "buildah", rename the docker backend to
  the container backend.

6 years agoUpdate the json docker file templates.
David Smith [Wed, 30 May 2018 16:24:32 +0000 (11:24 -0500)]
Update the json docker file templates.

* httpd/docker/centos.json: Go back to requiring systemtap-devel in the
  container.
* httpd/docker/fedora.json: Ditto.

6 years agoFix the boot time probing feature by updating dracut binary location.
Martin Cermak [Wed, 30 May 2018 11:54:21 +0000 (13:54 +0200)]
Fix the boot time probing feature by updating dracut binary location.

6 years agoMerge branch 'serhei/rt-fixes-clean'
Serhei Makarov [Tue, 29 May 2018 17:33:49 +0000 (13:33 -0400)]
Merge branch 'serhei/rt-fixes-clean'

Initial round of fixes for RHBZ1272304 to make systemtap work better on the
realtime (CONFIG_PREEMPT_RT_FULL) kernel. These fixes do not solve all of the
rule violations that occur (and get reported to dmesg on kernel-rt-debug) but
they do make it possible for SystemTap to get through the entire testsuite on
kernel-rt without locking up the system.

Merging rather than rebasing since only the final commit of the branch is 'safe'.

6 years agorhbz888262: add a %_systemtap_tapsetdir macro for add-on tapset packagers
Frank Ch. Eigler [Tue, 29 May 2018 15:08:06 +0000 (11:08 -0400)]
rhbz888262: add a %_systemtap_tapsetdir macro for add-on tapset packagers

6 years agoAdded ability to send test results via http to a url
Jafeer Uddin [Tue, 29 May 2018 13:28:17 +0000 (09:28 -0400)]
Added ability to send test results via http to a url

* testsuite/Makefile.am: Expanded check for DEJAZILLA to distinguish
  between email address and url, and send test results to the corresponding
  destination.
* testsuite/Makefile.in: Regenerated.
* testsuite/configure.ac: Updated messages to reflect the added feature.
* testsuite/configure: Regenerated.

6 years agoio_submit.stp: let the user know when the script is loaded
Jeff Moyer [Fri, 11 May 2018 19:25:52 +0000 (15:25 -0400)]
io_submit.stp: let the user know when the script is loaded

I often find myself checking lsmod to see when the script is finally
ready to collect data.  Just print a message from the begin probe to
make it obvious when the script is ready.

Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
6 years agoio_submit.stp: use an accumulator for traces
Jeff Moyer [Fri, 11 May 2018 19:25:51 +0000 (15:25 -0400)]
io_submit.stp: use an accumulator for traces

On very large systems, we get a lot of skipped probes due to lock
contention on the traces array.  The end result is that we don't
get any data for such systems.  Simply converting the traces array
to an accumulator resolves this issue in testing (on a highly-
loaded 288 cpu system).

Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
6 years agoio_submit.stp: fix array membership test
Jeff Moyer [Fri, 11 May 2018 19:25:50 +0000 (15:25 -0400)]
io_submit.stp: fix array membership test

io_submit.stp keeps track of threads which are currently executing
in the io_submit system call like so:

probe syscall.io_submit {
          in_iosubmit[tid()] = 1
}

probe syscall.io_submit.return {
          /* this assumes a given proc will do lots of io_submit calls, and
           * so doesn't do the more expensive "delete in_iosubmit[p]".  If
           * there are lots of procs doing small number of io_submit calls,
           * the hash may grow pretty big, so using delete may be better
           */
          in_iosubmit[tid()] = 0
}

However, the test to see if a thread is currently executing in io_submit
is performed using the membership operator 'in':

  if (tid() in in_iosubmit)

This is obviously wrong.  We can do one of two things:
1) change the test to if (in_iosubmit[tid()] == 1) or
2) just perform the delete in the return probe

While I agree that we typically have a small number of threads performing
io_submit, I don't believe there is substance to the performance claims
for the delete operator.  So, I've opted for solution 2.

Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
6 years agoUpdated spec file to change dependency on net-tools to iproute
Jafeer Uddin [Fri, 25 May 2018 14:42:41 +0000 (10:42 -0400)]
Updated spec file to change dependency on net-tools to iproute

6 years agorhbz1496156: removed dependencies on deprecated net-tools package
Jafeer Uddin [Thu, 24 May 2018 19:46:46 +0000 (15:46 -0400)]
rhbz1496156: removed dependencies on deprecated net-tools package

* java/stapbm.in: replaced netstat with ss

6 years agoChange the http docker backend to run systemtap in the container image.
David Smith [Wed, 23 May 2018 17:37:07 +0000 (12:37 -0500)]
Change the http docker backend to run systemtap in the container image.

* httpd/backends.cxx (docker_backend::generate_module): Switch back to
  running systemtap inside the container. Trying to use the container
  image as a sysroot didn't work well. Trying to run systemtap on the
  sysroot worked fine, but running gcc 8 (f28) against a centos 7
  sysroot's kernel source failed. Trying to run the centos 7 gcc from f28
  kept crashing. So, we're back to running systemtap in the container
  image.

6 years agokernel-rt fix: avoid use-after-free of pt_regs struct passed to task_work function. serhei/rt-fixes-clean
Serhei Makarov [Tue, 22 May 2018 20:06:59 +0000 (16:06 -0400)]
kernel-rt fix: avoid use-after-free of pt_regs struct passed to task_work function.

* stp_utrace.c (utrace_exec_work): move __stp_utrace_free_task_work() to avoid use-after-free.
(utrace_syscall_entry_work): ditto.

6 years agokernel-rt fix: only use stp_utrace.c task_work_queue on RT, remove excess printing
Serhei Makarov [Tue, 22 May 2018 20:02:34 +0000 (16:02 -0400)]
kernel-rt fix: only use stp_utrace.c task_work_queue on RT, remove excess printing

STP_UTRACE_USE_TASK_WORK_QUEUE doesn't *quite* work on non-RT kernels yet.

* stp_utrace.c: only define STP_UTRACE_USE_TASK_WORK_QUEUE on CONFIG_PREEMPT_RT_FULL.
(utrace_report_exec): comment out excess 'nonatomic context' printk.
(utrace_report_syscall_entry, utrace_report_syscall_exit): ditto.

6 years agorpm: by default, configure with --enable-docs=prebuilt
Frank Ch. Eigler [Wed, 16 May 2018 17:35:13 +0000 (13:35 -0400)]
rpm: by default, configure with --enable-docs=prebuilt

6 years agoFix testcases broken by not expectig 'WARNING: Missing unwind data'
Martin Cermak [Tue, 22 May 2018 11:59:35 +0000 (13:59 +0200)]
Fix testcases broken by not expectig 'WARNING: Missing unwind data'

* lib/stap_run.exp: Fix systemtap.base/caller.exp
* systemtap.base/backtrace.exp: Fix itself
* systemtap.context/backtrace.tcl: Fix systemtap.context/context.exp

6 years agoFix testcase systemtap.string/text_str.exp
Martin Cermak [Thu, 17 May 2018 08:51:18 +0000 (10:51 +0200)]
Fix testcase systemtap.string/text_str.exp

PR22991 (Invalid truncation of quoted strings) was fixed in
commit fa4f499d8c92600702853d763ae3ea45002c07e3, but respective
change hasn't been applied to systemtap.string/text_str.exp.
Adding the missing bit.

6 years agoFix testcase systemtap.unprivileged/unprivileged_embedded_C.exp
Martin Cermak [Wed, 16 May 2018 09:56:20 +0000 (11:56 +0200)]
Fix testcase systemtap.unprivileged/unprivileged_embedded_C.exp

Commit 2f6fcfc6 (Fix BZ1546179 by being careful when accessing
sk_buff structures) made the "unprivileged_embedded_C.exp"
testcase unhappy.  It started to report false positives.

Thing is that embeddedc.awk can't correctly parse two block
boundaries at a line, which is exactly what the commit introduces
by adding "} catch { }", or "try { ... } catch { }".  Here is the
failure as observed with git stap 0e051c5:

=======
FAIL: unprivileged embedded C: : --unprivileged: Unterminated block at: function _input_route_type:long (skb:long): 2 0
FAIL: unprivileged embedded C: : --privilege=stapusr: Unterminated block at EOF: 2 0
FAIL: unprivileged embedded C: : --privilege=stapsys: Unterminated block at: function _input_route_type:long (skb:long): 2 0
FAIL: unprivileged embedded C: : --privilege=stapsys: Unterminated block at EOF: 2 0
FAIL: unprivileged embedded C (transitive): : --privilege=stapusr: Unterminated block at: function _input_route_type:long (skb:long): 2 0
FAIL: unprivileged embedded C (transitive): : --privilege=stapsys: Unterminated block at: function _input_route_type:long (skb:long): 2 0
=======

This commit tries to address the issue by making embeddedc.awk
able to process two block boundaries at a line.

6 years agostapbpf: add foreach loop support
Aaron Merey [Fri, 11 May 2018 23:02:06 +0000 (19:02 -0400)]
stapbpf: add foreach loop support

* bpf-translate.cxx (visit_foreach_loop): translates foreach loops.

* bpf-internal.h (BPF_FUNC_map_get_next_key): add id for map_get_next_key.

* bpfinterp.cxx (bpf_interpret): call map_get_next_key when function id equals
  BPF_FUNC_map_get_next_key. (map_get_next_key): wrapper for bpf_get_next_key
  that includes support for limiting number of iterations and iterating in
  ascending/descending order.

* bpf_tests/foreach.stp: tests these changes.

6 years agoSend the target executable over to the http server.
David Smith [Mon, 7 May 2018 19:14:12 +0000 (14:14 -0500)]
Send the target executable over to the http server.

* httpd/backends.cxx (docker_backend::generate_module): Add the date and
  time to the docker image name, to make them easier to remove. Copy any
  target files from the zip file into the docker image.
* client-http.cxx (http_client::get_rpmname): If an rpm doesn't contain
  the file we're searching for, just return the empty string.
  (http_client_backend::include_file_or_directory): Only include the file
  or directory in the command line arguments if the callers asks.
  (http_client_backend::package_request): Move code from
  http_client_backend::find_and_connect_to_server(). If get_rpmname()
  returns the empty string, add the file so that it will get transferred
  over to the server.
  (http_client_backend::find_and_connect_to_server): Move code to
  http_client_backend::package_request().
* client-http.h (http_client_backend::include_file_or_directory): Updated
  prototype.
* client-nss.cxx (nss_client_backend::include_file_or_directory): Only
  include the file or directory in the command line arguments if the
  callers asks.
* client-nss.h (nss_client_backend::include_file_or_directory): Updated
  prototype.
* csclient.h: Updated client_backend::include_file_or_directory()
  prototype.

6 years agostapbpf: ensure uprobe DWARF offsets are 64-bits wide.
Aaron Merey [Fri, 4 May 2018 22:16:10 +0000 (18:16 -0400)]
stapbpf: ensure uprobe DWARF offsets are 64-bits wide.

stapbpf.cxx: change type of DWARF offset from unsigned long to unsigned long long.

6 years agoadd DEBUG_MEMALLOC_MIGHT_SLEEP option to check for memory allocations in RT-atomic
Serhei Makarov [Thu, 26 Apr 2018 20:34:36 +0000 (16:34 -0400)]
add DEBUG_MEMALLOC_MIGHT_SLEEP option to check for memory allocations in RT-atomic

If DEBUG_MEMALLOC_MIGHT_SLEEP is defined (stap -DDEBUG_MEMALLOC_MIGHT_SLEEP ...)
then all memory allocation operations are assumed to sleep. This
can be used to check whether any memory allocations occur in atomic
context, which is discouraged on realtime kernels.

* runtime/linux/alloc.c (DEBUG_MEMALLOC_MIGHT_SLEEP): new optional define.
(_stp_kmalloc_gfp, _stp_kmalloc_node_gfp, _stp_kzalloc_node_gfp, _stp_kfree,
 _stp_vfree, _stp_free_percpu): trigger might_sleep() on DEBUG_MEMALLOC_MIGHT_SLEEP.

6 years agokernel-rt temporary fix: flag pfiles.stp as incompatible with CONFIG_PREEMPT_RT_FULL
Serhei Makarov [Wed, 11 Apr 2018 18:47:27 +0000 (14:47 -0400)]
kernel-rt temporary fix: flag pfiles.stp as incompatible with CONFIG_PREEMPT_RT_FULL

testsuite/systemtap.examples/process/pfiles.stp: #error on CONFIG_PREEMPT_RT_FULL.

6 years agokernel-rt fix: delay cleanup of utrace struct after removal from task_utrace_table
Serhei Makarov [Wed, 25 Apr 2018 19:52:31 +0000 (15:52 -0400)]
kernel-rt fix: delay cleanup of utrace struct after removal from task_utrace_table

Since utrace->lock is now an ordinary spinlock, we cannot grab it in a
(raw) task_utrace_lock section, nor can we do memory management on kernel-rt
in a raw_spinlock section. Thus, in utrace_exit(), we dump utrace structures
to a separate hlist (guarded by a separate, non-raw utrace_cleanup_lock) before
calling utrace_cleanup() to free them.

* runtime/stp_utrace.c (utrace_cleanup_list, utrace_cleanup_lock): new data structures.
(utrace_exit): dump utrace structures to utrace_cleanup_list before releasing
task_utrace_lock and then actually cleaning up.
(utrace_cleanup): now requires utrace_cleanup_lock instead of task_utrace_lock.

6 years agokernel-rt fix: use task_work to defer reporting of stp_utrace events
Serhei Makarov [Wed, 11 Apr 2018 18:50:14 +0000 (14:50 -0400)]
kernel-rt fix: use task_work to defer reporting of stp_utrace events

Since utrace->lock is now a regular spinlock, it cannot be taken in tracepoint context.
This patch introduces a queue of task_work structures that the reporting functions
in stp_utrace.c can use to delay reporting work that requires taking utrace->lock.

* stp_utrace.c (STP_UTRACE_USE_TASK_WORK_QUEUE): new define.
 (__stp_utrace_task_work_list, __stp_utrace_task_work_list_lock,
__stp_utrace_task_work): new data structures.
(UTRACE_TASK_WORK_POOL_SIZE): new define, configure with -DUTRACE_TASK_WORK_POOL_SIZE
(__stp_utrace_task_work_pool_bitmap, __stp_utrace_task_work_pool,
__stp_utrace_task_work_pool_lock): new data structures.
(__stp_utrace_alloc_task_work): new function.
(__stp_utrace_save_regs): new function.
(__stp_utrace_free_task_work_from_pool): new function.
(__stp_utrace_free_task_work): new function.
(utrace_exit): free task_work_list items on exit.
(utrace_cancel_all_task_work): clean up code, also cancel task_work_list items.
(utrace_free): clean up code.
(utrace_report_exec): defer reporting to utrace_exec_work().
(utrace_exec_work): new function.
(utrace_report_syscall_entry): defer reporting to utrace_syscall_entry_work().
(utrace_syscall_entry_work): new function.
(utrace_report_syscall_exit): defer reporting to utrace_syscall_exit_work().
(utrace_syscall_exit_work): new function.
(utrace_report_clone): defer reporting to utrace_clone_work().
(utrace_clone_work): new function.

6 years agokernel-rt fix: change utrace->lock to a regular spinlock
Serhei Makarov [Mon, 26 Mar 2018 20:30:59 +0000 (16:30 -0400)]
kernel-rt fix: change utrace->lock to a regular spinlock

In utrace_control(), stp_lock_task_sighand() cannot be invoked in a raw-spinlock
section on kernel-RT. Therefore, we must change utrace->lock to a regular,
non-raw spinlock. (This change introduces additional issues to be fixed in subsequent
commits.)

* runtime/stp_utrace.c: switch to spinlock_t.

6 years agokernel-rt fix: avoid locking in __stp_time_init()
Serhei Makarov [Thu, 15 Mar 2018 20:26:55 +0000 (16:26 -0400)]
kernel-rt fix: avoid locking in __stp_time_init()

* runtime/time.c (stp_init_time): avoid invoking seqlock inside
an interrupts-disabled section.

The problem is that __stp_init_time() is invoked in an interrupts-disabled
section created by on_each_cpu(). This prevents it from being able to safely
grab seqlock.

My reasoning is that because __stp_time_init() is initializing the
seqlock, it is safe to assume no one else will be trying to grab it until
we return. So we can dispense with locking at this point.

6 years agokernel-rt fix: use stp_spinlock for _stp_ctl_ready_q
Serhei Makarov [Wed, 14 Mar 2018 19:42:08 +0000 (15:42 -0400)]
kernel-rt fix: use stp_spinlock for _stp_ctl_ready_q

_stp_ctl_ready_lock and _stp_ctl_special_msg_lock are locked within an atomic
context created by _stp_runtime_entryfn_get_context()/_put_context().
This is only legal on kernel-rt if the locks are raw stp_spinlocks.

* runtime/transport/control.h, runtime/transport/control.c: switch to stp_spinlock.
* runtime/transport/procfs.c: ditto.
* runtime/transport/transport.c: ditto.

This page took 0.068975 seconds and 5 git commands to generate.