## version info systemtap version: 4.0/0.176 kernel version: 3.10.0-1160.25.1.el7.x86_64 os: CentOS Linux 7 (Core) ```bash stap -V Systemtap translator/driver (version 4.0/0.176, rpm 4.0-13.el7) enabled features: AVAHI BOOST_STRING_REF DYNINST BPF JAVA PYTHON2 LIBRPM LIBSQLITE3 LIBVIRT LIBXML2 NLS NSS READLINE ``` ## what i want i'm try to probe nginx(has debuginfo) which running in docker. ## what i have do since nginx running in docker,so i just find the hostpath of the nginx in docker and run `stap -x 27368 -e 'probe process.syscall {printf("%s %d %s\n", execname(), pid(), pp())} probe process("/var/lib/docker/overlay2/{a-long-long-string}/merged/usr/local/openresty/nginx/sbin/nginx").function("*") {printf("%s",probefunc())}' `process.syscall` works find,but nginx probe just nothing happen. and `stap -x 27368 -e 'probe process.syscall {printf("%s %d %s\n", execname(), pid(), pp())} probe process("/usr/local/openresty/nginx/sbin/nginx").function("*") {printf("%s",probefunc())}' --sysroot=/var/lib/docker/overlay2/{a-long-long-string}/merged/ ` report err like `WARNING: probe process("/usr/local/openresty/nginx/sbin/nginx").function("ngx_write_fd@src/os/unix/ngx_files.h:145") inode-offset 0000000000041e55 registration error (rc -22)` ## what i searched i find some says [systemtap does not support overlays/union filesystems](https://stackoverflow.com/questions/57487292/systemtap-failed-to-probe-the-functions-registration-error) there is samiler question asked in stackoverflow,but have to meanning answer [smailer question but no answer](https://stackoverflow.com/questions/29314665/systemtapwhile-resolving-probe-point-identifier-process-at-source-probe-pro) [SystemTap_Beginners_Guide chapter 4 userspace-probing](https://www.sourceware.org/systemtap/SystemTap_Beginners_Guide/userspace-probing.html) just say you to point the exepath of the process. ## what i want to know is that true? how should i probe user process which run in docker? what should i know to fix this question? how to know what happen when uprobe just echo nothing? ## other info ```bash docker top a0bb57 UID PID PPID C STIME TTY TIME CMD root 20807 20755 0 11:19 ? 00:00:00 /sbin/tini -- /run.sh root 20852 20807 0 11:19 ? 00:00:00 sh /run.sh root 27368 20807 0 11:20 ? 00:00:00 nginx: master process /usr/local/openresty/nginx/sbin/nginx -c /usr/local/openresty/nginx/conf/nginx.conf root 27369 27368 4 11:20 ? 00:10:00 nginx: worker process root 27370 27368 0 11:20 ? 00:01:46 nginx: worker process ``` i have copy /lib/modules/3.10.0-1160.25.1.el7.x86_64/ to /var/lib/docker/overlay2/{a-long-long-string}/merged/
Consider trying the probe-by-buildid mechanism added recently: probe process("deadbeefdeadbeef").FOO { } instead of probe process("/path/to/binary").FOO { } ... where deadbeef is simply the hexadecimal buildid for the target binary, as printed by "file" or "readelf -n". Your RHEL7 version of stap is much too old for that, but maybe you can get hold of RH DTS builds, or build your own from upstream sources / RPMs, this should work. Otherwise, yeah, path name based matching is known to be difficult when there are many names for the same binary, and from different containers' points of view.
is there any way to distinguish between 1. systemtap does not probe to correct process (which should not happen because i set pid explicitly) 2. systemtap could not find those function which should probe 3. there is no event came out since the method 1 i use in before code does not report any wran/erorr ,it seems that it probe to correct process and find the debuginfo and find the function i want to probe. a correspond conjecture is that: "is just no event came out". did this logic chain make sense? or i just missing some point?
Running a stap module built with super verbosity like stap -DDEBUG_TASK_FINDER -DDEBUG_TASK_FINDER_VMA -DDEBUG_SYMBOLS -DDEBUG_UPROBES [...] may let you see the difference.