I use OpenSuSe 12.2 with XEN hypervisor. Studying the Systemtap (Chapter 4. User-Space Probing), I saw that it is possible to do probing for applications in user space, but it's not work in my kernel. Running the command below, everything works correctly. stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}' Pass 1: parsed user script and 90 library script(s) using 80812virt/24360res/2512shr/22300data kb, in 130usr/10sys/151real ms. Pass 2: analyzed script: 1 probe(s), 1 function(s), 3 embed(s), 0 global(s) using 300208virt/126464res/8372shr/115764data kb, in 1360usr/730sys/2131real ms. Pass 3: translated to C into "/tmp/stap7aWTJW/stap_6d420bdef22bdcac997a4dae1ea7f20b_1642_src.c" using 289736virt/121028res/5764shr/115764data kb, in 0usr/0sys/8real ms. Updating /tmp/stap7aWTJW/Makefile.xen Pass 4: compiled C into "stap_6d420bdef22bdcac997a4dae1ea7f20b_1642.ko" in 1430usr/630sys/2499real ms. Pass 5: starting run. read performed Pass 5: run completed in 10usr/50sys/403real ms. But, running the command below, nothing works. stap -e 'probe process("/lib64/libc-2.15.so").function("malloc") {println("test")}' user-space facilities not available without kernel CONFIG_UTRACE or CONFIG_TRACEPOINTS/CONFIG_ARCH_SUPPORTS_UPROBES/CONFIG_UPROBES Pass 4: compilation failed. Try again with another '--vp 0001' option. Can someone help me? Below my settings: Kernel (recompiled with debuginfo): 3.4.11-2.16-xen #3 SMP Tue Nov 27 19:00:52 BRST 2012 x86_64 x86_64 x86_64 GNU/Linux Systemtap: Systemtap translator/driver (version 2.1/0.153, commit release-2.0-108-g6b378b7) Copyright (C) 2005-2012 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS Packets DEBUG: # rpm -qa | grep debug glibc-locale-debuginfo-32bit-2.15-22.6.4.x86_64 glibc-locale-debuginfo-2.15-22.6.4.x86_64 kernel-debug-base-3.4.11-2.16.1.x86_64 glibc-devel-debuginfo-2.15-22.6.4.x86_64 glibc-devel-debuginfo-32bit-2.15-22.6.4.x86_64 glibc-utils-debuginfo-2.15-22.6.4.x86_64 kernel-debug-devel-3.4.11-2.16.1.x86_64 glibc-debuginfo-2.15-22.6.4.x86_64 build-initvm-debuginfo-i586-2012.07.19-1.1.1.x86_64 kmtrace-debuginfo-4.8.4-2.1.2.x86_64 glibc-debuginfo-32bit-2.15-22.6.4.x86_64 glibc-utils-debuginfo-32bit-2.15-22.6.4.x86_64 glibc-extra-debuginfo-2.15-22.6.4.x86_64
CONFIG_UPROBES-related code was merged into the Linus kernel only for version 3.5, so your 3.4.11 may simply be too old.
I installed the latest version of kernel (3.7.2), but the systemtap not working for probing in user space. I tested using the following command: stap -vvv -e 'probe process("/bin/ls").function("*") { log(pp()) }' -c /bin/ls Return of the command: Systemtap translator/driver (version 2.1/0.153, commit release-2.0-108-g6b378b7) Copyright (C) 2005-2012 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS Created temporary directory "/tmp/stapimWfUH" Session arch: x86_64 release: 3.7.2xen Parsed kernel "/lib/modules/3.7.2xen/build/.config", containing 4043 tuples Parsed kernel /lib/modules/3.7.2xen/build/Module.symvers, which contained 5975 vmlinux exports Searched for library macro files: "/usr/local/share/systemtap/tapset/linux/*.stpm", found: 1, processed: 1 Searched for library macro files: "/usr/local/share/systemtap/tapset/*.stpm", found: 1, processed: 1 Searched: "/usr/local/share/systemtap/tapset/linux/x86_64/*.stp", found: 3, processed: 3 Searched: "/usr/local/share/systemtap/tapset/linux/*.stp", found: 64, processed: 64 Searched: "/usr/local/share/systemtap/tapset/x86_64/*.stp", found: 1, processed: 1 Searched: "/usr/local/share/systemtap/tapset/*.stp", found: 20, processed: 20 Pass 1: parsed user script and 90 library script(s) using 81148virt/24524res/2524shr/22636data kb, in 120usr/10sys/137real ms. Extracting build ID. parse '*', func '*' focused on module '/usr/bin/ls' = [0x400000-0x61c368, bias 0 file /usr/bin/ls ELF machine |x86_64 (code 62) focused on module '/usr/bin/ls' WARNING: cannot find module /usr/bin/ls debuginfo: No DWARF information found semantic error: while resolving probe point: identifier 'process' at <input>:1:7 source: probe process("/bin/ls").function("*") { log(pp()) } ^ semantic error: no match Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 81788virt/25576res/2884shr/23164data kb, in 10usr/0sys/4real ms. Pass 2: analysis failed. Try again with another '--vp 01' option. Running rm -rf /tmp/stapimWfUH Spawn waitpid result (0x0): 0 Removed temporary directory "/tmp/stapimWfUH" Source of the code: http://sourceware.org/systemtap/wiki/HomePage?action=AttachFile&do=get&target=fosdem-stap.pdf Can someone help me?
(In reply to comment #2) > WARNING: cannot find module /usr/bin/ls debuginfo: No DWARF information found > semantic error: while resolving probe point: identifier 'process' at > <input>:1:7 > source: probe process("/bin/ls").function("*") { log(pp()) } > ^ > > semantic error: no match SystemTap needs debug information to locate functions in your binary, which you can install with your distro's package management. I'm not sure for openSUSE, but in Fedora the package for /usr/bin/ls would be coreutils-debuginfo. Note also, the version of the debuginfo package must exactly match the version of the package which provided /usr/bin/ls.
Correct Josh, but if I build my code with debug it should work right? In my case, this did not work. Look: My code is (Command for compile is: gcc -g -o Application Application.c): #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <time.h> int TimeForSleep(); int TimeForSleep(){ double nr=3.0; return 1000000 * nr; } int main(int argc,char *argv[]){ while(1){ printf("test \n"); usleep(TimeForSleep()); } return 0; } My script of systemtap: probe process("/home/elder/Application").function("TimeForSleep").call { printf("found\n"); } probe begin { printf("INI\n"); } probe end { printf("FIM \n"); } Return of systemtap: # stap -vvv s.stap -c /home/elder/Application Systemtap translator/driver (version 2.1/0.153, commit release-2.0-219-gf5d0bf6) Copyright (C) 2005-2013 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS Created temporary directory "/tmp/stapBigAmY" Session arch: x86_64 release: 3.7.2xen Parsed kernel "/lib/modules/3.7.2xen/build/.config", containing 4043 tuples Parsed kernel /lib/modules/3.7.2xen/build/Module.symvers, which contained 5975 vmlinux exports Processing tapset "/usr/local/share/systemtap/tapset/linux/context.stpm" Searched for library macro files: "/usr/local/share/systemtap/tapset/linux/*.stpm", found: 1, processed: 1 Processing tapset "/usr/local/share/systemtap/tapset/choose_defined.stpm" Searched for library macro files: "/usr/local/share/systemtap/tapset/*.stpm", found: 1, processed: 1 Processing tapset "/usr/local/share/systemtap/tapset/linux/x86_64/aux_syscalls.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/x86_64/nd_syscalls.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/x86_64/syscalls.stp" Searched: "/usr/local/share/systemtap/tapset/linux/x86_64/*.stp", found: 3, processed: 3 Processing tapset "/usr/local/share/systemtap/tapset/linux/atomic.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/aux_syscalls.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/context-caller.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/context-envvar.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/context.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/context-symbols.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/context-unwind.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/conversions-guru.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/conversions.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ctime.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/dentry.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/dev.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/endian.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/guru-delay.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/inet_sock.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/inet.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ioblock.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ioscheduler.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ipmib-filter-default.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ipmib.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ip.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/irq.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/kprocess.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/kretprobe.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/linuxmib-filter-default.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/linuxmib.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/logging.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/memory.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/nd_syscalls2.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/nd_syscalls.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/netfilter.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/networking.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/nfsderrno.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/nfsd.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/nfs_proc.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/nfs.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/panic.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/perf.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/proc_mem.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/pstrace.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/rpc.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/scheduler.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/scsi.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/signal.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/socket.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/syscalls2.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/syscalls_cfg_trunc.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/syscalls.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/target_set.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/task.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/task_time.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/tcpmib-filter-default.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/tcpmib.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/tcp.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/timestamp_gtod.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/timestamp_monotonic.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/timestamp.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/tty.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ucontext.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ucontext-symbols.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/ucontext-unwind.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/udp.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/utrace.stp" Processing tapset "/usr/local/share/systemtap/tapset/linux/vfs.stp" Searched: "/usr/local/share/systemtap/tapset/linux/*.stp", found: 64, processed: 64 Processing tapset "/usr/local/share/systemtap/tapset/x86_64/registers.stp" Searched: "/usr/local/share/systemtap/tapset/x86_64/*.stp", found: 1, processed: 1 Processing tapset "/usr/local/share/systemtap/tapset/ansi.stp" Processing tapset "/usr/local/share/systemtap/tapset/argv.stp" Processing tapset "/usr/local/share/systemtap/tapset/context.stp" Processing tapset "/usr/local/share/systemtap/tapset/errno.stp" Processing tapset "/usr/local/share/systemtap/tapset/indent-default.stp" Processing tapset "/usr/local/share/systemtap/tapset/indent.stp" Processing tapset "/usr/local/share/systemtap/tapset/logging.stp" Processing tapset "/usr/local/share/systemtap/tapset/null.stp" Processing tapset "/usr/local/share/systemtap/tapset/pn.stp" Processing tapset "/usr/local/share/systemtap/tapset/queue_stats.stp" Processing tapset "/usr/local/share/systemtap/tapset/random.stp" Processing tapset "/usr/local/share/systemtap/tapset/registers.stp" Processing tapset "/usr/local/share/systemtap/tapset/speculative.stp" Processing tapset "/usr/local/share/systemtap/tapset/stap_staticmarkers.stp" Processing tapset "/usr/local/share/systemtap/tapset/stopwatch.stp" Processing tapset "/usr/local/share/systemtap/tapset/string.stp" Processing tapset "/usr/local/share/systemtap/tapset/system.stp" Processing tapset "/usr/local/share/systemtap/tapset/tokenize.stp" Processing tapset "/usr/local/share/systemtap/tapset/tzinfo.stp" Processing tapset "/usr/local/share/systemtap/tapset/uconversions.stp" Searched: "/usr/local/share/systemtap/tapset/*.stp", found: 20, processed: 20 Pass 1: parsed user script and 90 library script(s) using 81316virt/24580res/2532shr/22780data kb, in 140usr/20sys/194real ms. Extracting build ID. parse 'TimeForSleep', func 'TimeForSleep' focused on module '/home/elder/Application' = [0x400000-0x601030, bias 0 file /home/elder/Application ELF machine |x86_64 (code 62) focused on module '/home/elder/Application' selected function TimeForSleep selected function TimeForSleep searching for prologue of function 'TimeForSleep' 0x40058c-0x4005b5@/home/elder/Application.c:6 checking line record 0x40058c@/home/elder/Application.c:6 checking line record 0x400590@/home/elder/Application.c:7 prologue found function 'TimeForSleep' = 0x400590 probe TimeForSleep@/home/elder/Application.c:6 process=/home/elder/Application reloc=.absolute pc=0x400590 Eliding side-effect-free singleton block operator '{' at s.stap:2:1 Eliding side-effect-free singleton block operator '{' at s.stap:5:13 Eliding side-effect-free singleton block operator '{' at s.stap:8:11 Pass 2: analyzed script: 3 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 81860virt/25636res/2852shr/23308data kb, in 0usr/0sys/6real ms. function recursion-analysis: max-nesting 0 non-recursive probe process("/home/elder/Application").function("TimeForSleep@/home/elder/Application.c:6").call locks nothing adapt location:13max - 39/3tot =>[] adapt derivation:78max - 106/3tot =>* adapt pp:99max - 113/3tot =>* adapt pn:99max - 113/3tot =>* dump_unwindsyms /home/elder/Application index=0 base=0x400000 Found build-id in /home/elder/Application, length 20, start at 0x400284 Pass 3: translated to C into "/tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.c" using 217176virt/26328res/3376shr/23388data kb, in 40usr/360sys/394real ms. Pass 4: using cached /root/.systemtap/cache/d9/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h Running env -uARCH -uKBUILD_EXTMOD -uCROSS_COMPILE -uKBUILD_IMAGE -uKCONFIG_CONFIG -uINSTALL_PATH PATH=/usr/bin:/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/sbin make -C /lib/modules/3.7.2xen/build M=/tmp/stapBigAmY modules ARCH=x86_64 CONFIG_DEBUG_INFO= V=1 -j5 make: Entering directory `/home/elder/linux-3.7.2' test -e include/generated/autoconf.h -a -e include/config/auto.conf || ( \ echo >&2; \ echo >&2 " ERROR: Kernel configuration is invalid."; \ echo >&2 " include/generated/autoconf.h or include/config/auto.conf are missing.";\ echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix it."; \ echo >&2 ; \ /bin/false) mkdir -p /tmp/stapBigAmY/.tmp_versions ; rm -f /tmp/stapBigAmY/.tmp_versions/* make -f scripts/Makefile.build obj=/tmp/stapBigAmY (cat /dev/null; echo kernel//tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko;) > /tmp/stapBigAmY/modules.order gcc -Wp,-MD,/tmp/stapBigAmY/.stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.o.d -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.7/include -I/home/elder/linux-3.7.2/arch/x86/include -Iarch/x86/include/generated -Iinclude -I/home/elder/linux-3.7.2/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/elder/linux-3.7.2/include/uapi -Iinclude/generated/uapi -include /home/elder/linux-3.7.2/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -Iinclude2/asm/mach-default -include /tmp/stapBigAmY/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h -freorder-blocks -Wframe-larger-than=256 -Wno-unused -Werror -I"/usr/local/share/systemtap/runtime" -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src)" -D"KBUILD_MODNAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045)" -c -o /tmp/stapBigAmY/.tmp_stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.c ld -m elf_x86_64 -r -o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.o ; scripts/mod/modpost /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.o make -f /home/elder/linux-3.7.2/scripts/Makefile.modpost scripts/mod/modpost -m -a -i /home/elder/linux-3.7.2/Module.symvers -I /tmp/stapBigAmY/Module.symvers -o /tmp/stapBigAmY/Module.symvers -w -s gcc -Wp,-MD,/tmp/stapBigAmY/.stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.o.d -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.7/include -I/home/elder/linux-3.7.2/arch/x86/include -Iarch/x86/include/generated -Iinclude -I/home/elder/linux-3.7.2/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/elder/linux-3.7.2/include/uapi -Iinclude/generated/uapi -include /home/elder/linux-3.7.2/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -Iinclude2/asm/mach-default -include /tmp/stapBigAmY/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h -freorder-blocks -Wframe-larger-than=256 -Wno-unused -Werror -I"/usr/local/share/systemtap/runtime" -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod)" -D"KBUILD_MODNAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045)" -DMODULE -c -o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.c ld -r -m elf_x86_64 -T /home/elder/linux-3.7.2/scripts/module-common.lds --build-id -o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.o make: Leaving directory `/home/elder/linux-3.7.2' Spawn waitpid result (0x0): 0 Pass 4: compiled C into "stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko" in 2110usr/610sys/2868real ms. Cleaning cache, interval reached 614 s > 300 s. Copying /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko to /root/.systemtap/cache/c3/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko Copying /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.c to /root/.systemtap/cache/c3/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.c Copying /tmp/stapBigAmY/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h to /root/.systemtap/cache/d9/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h Pass 5: starting run. Running /usr/local/bin/staprun -v -v -c /home/elder/Application -R /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko staprun:main:412 modpath="/tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko", modname="stap_c35cc695695a4d1cc021c5f060b93ddb_1045" staprun:init_staprun:314 init_staprun staprun:insert_module:73 inserting module /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko staprun:insert_module:99 module options: _stp_bufsize=0 staprun:insert_module:107 module path canonicalized to '/tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko' staprun:check_signature:431 checking signature for /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko Signature file /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko.sgn not found staprun:check_signature:444 verify_module returns 0 staprun:insert_module:170 Renaming module 'stap_c35cc695695a4d1cc021c5f060b93ddb_1045' staprun:insert_module:178 Renamed module to 'stap_c35cc695695a4d1cc021c5f060b93ddb__929' staprun:insert_module:185 Module stap_c35cc695695a4d1cc021c5f060b93ddb__929 inserted from file /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko staprun:init_ctl_channel:72 Opened /sys/kernel/debug/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929/.cmd (4) staprun:close_ctl_channel:112 Closed ctl fd 4 execing: /usr/local/lib/systemtap/stapio -v -v -c /home/elder/Application -R stap_c35cc695695a4d1cc021c5f060b93ddb__929 -F3 stapio:parse_modpath:425 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko" stapio:main:50 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko", modname="stap_c35cc695695a4d1cc021c5f060b93ddb__929" stapio:init_stapio:369 init_stapio stapio:init_ctl_channel:30 Opened .cmd (4) stapio:stp_main_loop:567 in main loop stapio:stp_main_loop:586 select_supported: 1 stapio:init_relayfs:244 initializing relayfs stapio:init_relayfs:269 stapio:start_cmd:249 attempting to open trace0 blocking briefly stapio:init_relayfs:269 attempting to open trace1 stapio:init_relayfs:281 ncpus=1, bulkmode = 0 stapio:init_relayfs:364 starting threads stapio:stp_main_loop:738 systemtap_module_init() returned 0 stapio:stp_main_loop:744 detaching pid 930 stapio:start_cmd:268 execing target_cmd /home/elder/Application test stapio:chld_proc:71 chld_proc 17 (Child exited) Warning: child process exited with signal 11 (Segmentation fault) INI FIM stapio:stp_main_loop:729 got STP_REQUEST_EXIT stapio:stp_main_loop:722 got STP_EXIT stapio:cleanup_and_exit:454 detach=0 stapio:close_relayfs:380 closing stapio:close_relayfs:399 done stapio:cleanup_and_exit:471 closing control channel stapio:close_ctl_channel:112 Closed ctl fd 4 stapio:cleanup_and_exit:479 Renamed module to: stap_c35cc695695a4d1cc021c5f060b93ddb__929 stapio:cleanup_and_exit:486 removing stap_c35cc695695a4d1cc021c5f060b93ddb__929 staprun:parse_modpath:425 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko" staprun:main:412 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko", modname="stap_c35cc695695a4d1cc021c5f060b93ddb__929" staprun:init_staprun:314 init_staprun staprun:remove_module:225 stap_c35cc695695a4d1cc021c5f060b93ddb__929 staprun:init_ctl_channel:72 Opened /sys/kernel/debug/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929/.cmd (4) staprun:close_ctl_channel:112 Closed ctl fd 4 staprun:remove_module:250 removing module stap_c35cc695695a4d1cc021c5f060b93ddb__929 staprun:remove_module:262 Module stap_c35cc695695a4d1cc021c5f060b93ddb__929 removed. Spawn waitpid result (0x100): 1 WARNING: /usr/local/bin/staprun exited with status: 1 Pass 5: run completed in 10usr/50sys/328real ms. Pass 5: run failed. Try again with another '--vp 00001' option. Running rm -rf /tmp/stapBigAmY Spawn waitpid result (0x0): 0 Removed temporary directory "/tmp/stapBigAmY" My dmesg: [ 9211.351238] stap_c35cc695695a4d1cc021c5f060b93ddb__929: systemtap: 2.1/0.153, base: ffffffffa062f000, memory: 55data/40text/2ctx/2058net/33alloc kb, probes: 3 [ 9211.354265] Application[930]: segfault at 0 ip 0000000000400596 sp 00007ffff8932d30 error 6 in Application[400000+1000]
This sounds like a kernel uprobes bug, perhaps related to xen.
current kernels seem to generally work