Bug 15016 - kernel 3.7.2xen uprobes can cause SEGV on probed app
Summary: kernel 3.7.2xen uprobes can cause SEGV on probed app
Status: RESOLVED OBSOLETE
Alias: None
Product: systemtap
Classification: Unclassified
Component: uprobes (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-14 21:46 UTC by Elder
Modified: 2020-04-24 17:10 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Elder 2013-01-14 21:46:16 UTC
I use OpenSuSe 12.2 with XEN hypervisor. Studying the Systemtap (Chapter 4. User-Space Probing), I saw that it is possible to do probing for applications in user space, but it's not work in my kernel.

Running the command below, everything works correctly.
stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}' 
Pass 1: parsed user script and 90 library script(s) using 80812virt/24360res/2512shr/22300data kb, in 130usr/10sys/151real ms.
Pass 2: analyzed script: 1 probe(s), 1 function(s), 3 embed(s), 0 global(s) using 300208virt/126464res/8372shr/115764data kb, in 1360usr/730sys/2131real ms.
Pass 3: translated to C into "/tmp/stap7aWTJW/stap_6d420bdef22bdcac997a4dae1ea7f20b_1642_src.c" using 289736virt/121028res/5764shr/115764data kb, in 0usr/0sys/8real ms.
Updating /tmp/stap7aWTJW/Makefile.xen
Pass 4: compiled C into "stap_6d420bdef22bdcac997a4dae1ea7f20b_1642.ko" in 1430usr/630sys/2499real ms.
Pass 5: starting run.
read performed
Pass 5: run completed in 10usr/50sys/403real ms.




But, running the command below, nothing works.
stap -e 'probe process("/lib64/libc-2.15.so").function("malloc") {println("test")}'
user-space facilities not available without kernel CONFIG_UTRACE or CONFIG_TRACEPOINTS/CONFIG_ARCH_SUPPORTS_UPROBES/CONFIG_UPROBES
Pass 4: compilation failed. Try again with another '--vp 0001' option.




Can someone help me?



Below my settings:
Kernel (recompiled with debuginfo):
3.4.11-2.16-xen #3 SMP Tue Nov 27 19:00:52 BRST 2012 x86_64 x86_64 x86_64 GNU/Linux

Systemtap:
Systemtap translator/driver (version 2.1/0.153, commit release-2.0-108-g6b378b7)
Copyright (C) 2005-2012 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS

Packets DEBUG: # rpm -qa | grep debug
glibc-locale-debuginfo-32bit-2.15-22.6.4.x86_64
glibc-locale-debuginfo-2.15-22.6.4.x86_64
kernel-debug-base-3.4.11-2.16.1.x86_64
glibc-devel-debuginfo-2.15-22.6.4.x86_64
glibc-devel-debuginfo-32bit-2.15-22.6.4.x86_64
glibc-utils-debuginfo-2.15-22.6.4.x86_64
kernel-debug-devel-3.4.11-2.16.1.x86_64
glibc-debuginfo-2.15-22.6.4.x86_64
build-initvm-debuginfo-i586-2012.07.19-1.1.1.x86_64
kmtrace-debuginfo-4.8.4-2.1.2.x86_64
glibc-debuginfo-32bit-2.15-22.6.4.x86_64
glibc-utils-debuginfo-32bit-2.15-22.6.4.x86_64
glibc-extra-debuginfo-2.15-22.6.4.x86_64
Comment 1 Frank Ch. Eigler 2013-01-14 21:54:40 UTC
CONFIG_UPROBES-related code was merged into the Linus kernel only for version 3.5, so your 3.4.11 may simply be too old.
Comment 2 Elder 2013-01-15 23:11:00 UTC
I installed the latest version of kernel (3.7.2), but the systemtap not working for probing in user space.

I tested using the following command:

stap -vvv -e 'probe process("/bin/ls").function("*") { log(pp()) }' -c /bin/ls



Return of the command:

Systemtap translator/driver (version 2.1/0.153, commit release-2.0-108-g6b378b7)
Copyright (C) 2005-2012 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS
Created temporary directory "/tmp/stapimWfUH"
Session arch: x86_64 release: 3.7.2xen
Parsed kernel "/lib/modules/3.7.2xen/build/.config", containing 4043 tuples
Parsed kernel /lib/modules/3.7.2xen/build/Module.symvers, which contained 5975 vmlinux exports
Searched for library macro files: "/usr/local/share/systemtap/tapset/linux/*.stpm", found: 1, processed: 1
Searched for library macro files: "/usr/local/share/systemtap/tapset/*.stpm", found: 1, processed: 1
Searched: "/usr/local/share/systemtap/tapset/linux/x86_64/*.stp", found: 3, processed: 3
Searched: "/usr/local/share/systemtap/tapset/linux/*.stp", found: 64, processed: 64
Searched: "/usr/local/share/systemtap/tapset/x86_64/*.stp", found: 1, processed: 1
Searched: "/usr/local/share/systemtap/tapset/*.stp", found: 20, processed: 20
Pass 1: parsed user script and 90 library script(s) using 81148virt/24524res/2524shr/22636data kb, in 120usr/10sys/137real ms.
Extracting build ID.
parse '*', func '*'
focused on module '/usr/bin/ls' = [0x400000-0x61c368, bias 0 file /usr/bin/ls ELF machine |x86_64 (code 62)
focused on module '/usr/bin/ls'
WARNING: cannot find module /usr/bin/ls debuginfo: No DWARF information found
semantic error: while resolving probe point: identifier 'process' at <input>:1:7
        source: probe process("/bin/ls").function("*") { log(pp()) }
                      ^

semantic error: no match
Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 81788virt/25576res/2884shr/23164data kb, in 10usr/0sys/4real ms.
Pass 2: analysis failed.  Try again with another '--vp 01' option.
Running rm -rf /tmp/stapimWfUH
Spawn waitpid result (0x0): 0
Removed temporary directory "/tmp/stapimWfUH"


Source of the code: http://sourceware.org/systemtap/wiki/HomePage?action=AttachFile&do=get&target=fosdem-stap.pdf


Can someone help me?
Comment 3 Josh Stone 2013-01-16 00:02:07 UTC
(In reply to comment #2)
> WARNING: cannot find module /usr/bin/ls debuginfo: No DWARF information found
> semantic error: while resolving probe point: identifier 'process' at
> <input>:1:7
>         source: probe process("/bin/ls").function("*") { log(pp()) }
>                       ^
> 
> semantic error: no match

SystemTap needs debug information to locate functions in your binary, which you can install with your distro's package management.  I'm not sure for openSUSE, but in Fedora the package for /usr/bin/ls would be coreutils-debuginfo.

Note also, the version of the debuginfo package must exactly match the version of the package which provided /usr/bin/ls.
Comment 4 Elder 2013-01-16 16:49:59 UTC
Correct Josh, but if I build my code with debug it should work right?
In my case, this did not work. Look:

My code is (Command for compile is: gcc -g -o Application Application.c):
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <time.h>
int TimeForSleep();
int TimeForSleep(){
  double nr=3.0;
  return 1000000 * nr;
}
int main(int argc,char *argv[]){
  while(1){
    printf("test \n");
    usleep(TimeForSleep()); 
  }
  return 0;
}


My script of systemtap:
probe process("/home/elder/Application").function("TimeForSleep").call 
{
 printf("found\n"); 
}
probe begin {
 printf("INI\n"); 
}
probe end {
 printf("FIM \n"); 
}



Return of systemtap:
# stap -vvv s.stap -c /home/elder/Application
Systemtap translator/driver (version 2.1/0.153, commit release-2.0-219-gf5d0bf6)
Copyright (C) 2005-2013 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS
Created temporary directory "/tmp/stapBigAmY"
Session arch: x86_64 release: 3.7.2xen
Parsed kernel "/lib/modules/3.7.2xen/build/.config", containing 4043 tuples
Parsed kernel /lib/modules/3.7.2xen/build/Module.symvers, which contained 5975 vmlinux exports
Processing tapset "/usr/local/share/systemtap/tapset/linux/context.stpm"
Searched for library macro files: "/usr/local/share/systemtap/tapset/linux/*.stpm", found: 1, processed: 1
Processing tapset "/usr/local/share/systemtap/tapset/choose_defined.stpm"
Searched for library macro files: "/usr/local/share/systemtap/tapset/*.stpm", found: 1, processed: 1
Processing tapset "/usr/local/share/systemtap/tapset/linux/x86_64/aux_syscalls.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/x86_64/nd_syscalls.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/x86_64/syscalls.stp"
Searched: "/usr/local/share/systemtap/tapset/linux/x86_64/*.stp", found: 3, processed: 3
Processing tapset "/usr/local/share/systemtap/tapset/linux/atomic.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/aux_syscalls.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/context-caller.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/context-envvar.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/context.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/context-symbols.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/context-unwind.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/conversions-guru.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/conversions.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ctime.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/dentry.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/dev.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/endian.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/guru-delay.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/inet_sock.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/inet.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ioblock.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ioscheduler.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ipmib-filter-default.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ipmib.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ip.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/irq.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/kprocess.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/kretprobe.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/linuxmib-filter-default.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/linuxmib.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/logging.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/memory.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/nd_syscalls2.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/nd_syscalls.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/netfilter.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/networking.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/nfsderrno.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/nfsd.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/nfs_proc.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/nfs.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/panic.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/perf.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/proc_mem.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/pstrace.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/rpc.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/scheduler.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/scsi.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/signal.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/socket.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/syscalls2.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/syscalls_cfg_trunc.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/syscalls.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/target_set.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/task.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/task_time.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/tcpmib-filter-default.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/tcpmib.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/tcp.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/timestamp_gtod.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/timestamp_monotonic.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/timestamp.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/tty.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ucontext.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ucontext-symbols.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/ucontext-unwind.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/udp.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/utrace.stp"
Processing tapset "/usr/local/share/systemtap/tapset/linux/vfs.stp"
Searched: "/usr/local/share/systemtap/tapset/linux/*.stp", found: 64, processed: 64
Processing tapset "/usr/local/share/systemtap/tapset/x86_64/registers.stp"
Searched: "/usr/local/share/systemtap/tapset/x86_64/*.stp", found: 1, processed: 1
Processing tapset "/usr/local/share/systemtap/tapset/ansi.stp"
Processing tapset "/usr/local/share/systemtap/tapset/argv.stp"
Processing tapset "/usr/local/share/systemtap/tapset/context.stp"
Processing tapset "/usr/local/share/systemtap/tapset/errno.stp"
Processing tapset "/usr/local/share/systemtap/tapset/indent-default.stp"
Processing tapset "/usr/local/share/systemtap/tapset/indent.stp"
Processing tapset "/usr/local/share/systemtap/tapset/logging.stp"
Processing tapset "/usr/local/share/systemtap/tapset/null.stp"
Processing tapset "/usr/local/share/systemtap/tapset/pn.stp"
Processing tapset "/usr/local/share/systemtap/tapset/queue_stats.stp"
Processing tapset "/usr/local/share/systemtap/tapset/random.stp"
Processing tapset "/usr/local/share/systemtap/tapset/registers.stp"
Processing tapset "/usr/local/share/systemtap/tapset/speculative.stp"
Processing tapset "/usr/local/share/systemtap/tapset/stap_staticmarkers.stp"
Processing tapset "/usr/local/share/systemtap/tapset/stopwatch.stp"
Processing tapset "/usr/local/share/systemtap/tapset/string.stp"
Processing tapset "/usr/local/share/systemtap/tapset/system.stp"
Processing tapset "/usr/local/share/systemtap/tapset/tokenize.stp"
Processing tapset "/usr/local/share/systemtap/tapset/tzinfo.stp"
Processing tapset "/usr/local/share/systemtap/tapset/uconversions.stp"
Searched: "/usr/local/share/systemtap/tapset/*.stp", found: 20, processed: 20
Pass 1: parsed user script and 90 library script(s) using 81316virt/24580res/2532shr/22780data kb, in 140usr/20sys/194real ms.
Extracting build ID.
parse 'TimeForSleep', func 'TimeForSleep'
focused on module '/home/elder/Application' = [0x400000-0x601030, bias 0 file /home/elder/Application ELF machine |x86_64 (code 62)
focused on module '/home/elder/Application'
selected function TimeForSleep
selected function TimeForSleep
searching for prologue of function 'TimeForSleep' 0x40058c-0x4005b5@/home/elder/Application.c:6
checking line record 0x40058c@/home/elder/Application.c:6
checking line record 0x400590@/home/elder/Application.c:7
prologue found function 'TimeForSleep' = 0x400590
probe TimeForSleep@/home/elder/Application.c:6 process=/home/elder/Application reloc=.absolute pc=0x400590
Eliding side-effect-free singleton block operator '{' at s.stap:2:1
Eliding side-effect-free singleton block operator '{' at s.stap:5:13
Eliding side-effect-free singleton block operator '{' at s.stap:8:11
Pass 2: analyzed script: 3 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 81860virt/25636res/2852shr/23308data kb, in 0usr/0sys/6real ms.
function recursion-analysis: max-nesting 0  non-recursive
probe process("/home/elder/Application").function("TimeForSleep@/home/elder/Application.c:6").call locks nothing
adapt location:13max - 39/3tot =>[]
adapt derivation:78max - 106/3tot =>*
adapt pp:99max - 113/3tot =>*
adapt pn:99max - 113/3tot =>*
dump_unwindsyms /home/elder/Application index=0 base=0x400000
Found build-id in /home/elder/Application, length 20, start at 0x400284
Pass 3: translated to C into "/tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.c" using 217176virt/26328res/3376shr/23388data kb, in 40usr/360sys/394real ms.
Pass 4: using cached /root/.systemtap/cache/d9/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h
Running env -uARCH -uKBUILD_EXTMOD -uCROSS_COMPILE -uKBUILD_IMAGE -uKCONFIG_CONFIG -uINSTALL_PATH PATH=/usr/bin:/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/sbin make -C /lib/modules/3.7.2xen/build M=/tmp/stapBigAmY modules ARCH=x86_64 CONFIG_DEBUG_INFO= V=1 -j5
make: Entering directory `/home/elder/linux-3.7.2'
test -e include/generated/autoconf.h -a -e include/config/auto.conf || (                \
echo >&2;                                                       \
echo >&2 "  ERROR: Kernel configuration is invalid.";           \
echo >&2 "         include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 "         Run 'make oldconfig && make prepare' on kernel src to fix it.";      \
echo >&2 ;                                                      \
/bin/false)
mkdir -p /tmp/stapBigAmY/.tmp_versions ; rm -f /tmp/stapBigAmY/.tmp_versions/*
make -f scripts/Makefile.build obj=/tmp/stapBigAmY
(cat /dev/null;   echo kernel//tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko;) > /tmp/stapBigAmY/modules.order
  gcc -Wp,-MD,/tmp/stapBigAmY/.stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.o.d  -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.7/include -I/home/elder/linux-3.7.2/arch/x86/include -Iarch/x86/include/generated  -Iinclude -I/home/elder/linux-3.7.2/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/elder/linux-3.7.2/include/uapi -Iinclude/generated/uapi -include /home/elder/linux-3.7.2/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -Iinclude2/asm/mach-default -include /tmp/stapBigAmY/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h -freorder-blocks -Wframe-larger-than=256 -Wno-unused -Werror -I"/usr/local/share/systemtap/runtime"  -DMODULE  -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src)"  -D"KBUILD_MODNAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045)" -c -o /tmp/stapBigAmY/.tmp_stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.c
  ld -m elf_x86_64   -r -o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.o ; scripts/mod/modpost /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.o
make -f /home/elder/linux-3.7.2/scripts/Makefile.modpost
  scripts/mod/modpost -m -a -i /home/elder/linux-3.7.2/Module.symvers -I /tmp/stapBigAmY/Module.symvers  -o /tmp/stapBigAmY/Module.symvers  -w  -s
  gcc -Wp,-MD,/tmp/stapBigAmY/.stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.o.d  -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.7/include -I/home/elder/linux-3.7.2/arch/x86/include -Iarch/x86/include/generated  -Iinclude -I/home/elder/linux-3.7.2/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/elder/linux-3.7.2/include/uapi -Iinclude/generated/uapi -include /home/elder/linux-3.7.2/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -Iinclude2/asm/mach-default -include /tmp/stapBigAmY/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h -freorder-blocks -Wframe-larger-than=256 -Wno-unused -Werror -I"/usr/local/share/systemtap/runtime"  -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod)"  -D"KBUILD_MODNAME=KBUILD_STR(stap_c35cc695695a4d1cc021c5f060b93ddb_1045)" -DMODULE  -c -o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.c
  ld -r -m elf_x86_64 -T /home/elder/linux-3.7.2/scripts/module-common.lds --build-id  -o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.o /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.mod.o
make: Leaving directory `/home/elder/linux-3.7.2'
Spawn waitpid result (0x0): 0
Pass 4: compiled C into "stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko" in 2110usr/610sys/2868real ms.
Cleaning cache, interval reached 614 s > 300 s.
Copying /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko to /root/.systemtap/cache/c3/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko
Copying /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045_src.c to /root/.systemtap/cache/c3/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.c
Copying /tmp/stapBigAmY/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h to /root/.systemtap/cache/d9/stapconf_d9cd427a508d75332e973f88a9e2f489_654.h
Pass 5: starting run.
Running /usr/local/bin/staprun -v -v -c /home/elder/Application -R /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko
staprun:main:412 modpath="/tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko", modname="stap_c35cc695695a4d1cc021c5f060b93ddb_1045"
staprun:init_staprun:314 init_staprun
staprun:insert_module:73 inserting module /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko
staprun:insert_module:99 module options: _stp_bufsize=0
staprun:insert_module:107 module path canonicalized to '/tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko'
staprun:check_signature:431 checking signature for /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko
Signature file /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko.sgn not found
staprun:check_signature:444 verify_module returns 0
staprun:insert_module:170 Renaming module 'stap_c35cc695695a4d1cc021c5f060b93ddb_1045'
staprun:insert_module:178 Renamed module to 'stap_c35cc695695a4d1cc021c5f060b93ddb__929'
staprun:insert_module:185 Module stap_c35cc695695a4d1cc021c5f060b93ddb__929 inserted from file /tmp/stapBigAmY/stap_c35cc695695a4d1cc021c5f060b93ddb_1045.ko
staprun:init_ctl_channel:72 Opened /sys/kernel/debug/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929/.cmd (4)
staprun:close_ctl_channel:112 Closed ctl fd 4
execing: /usr/local/lib/systemtap/stapio -v -v -c /home/elder/Application -R stap_c35cc695695a4d1cc021c5f060b93ddb__929 -F3 
stapio:parse_modpath:425 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko"
stapio:main:50 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko", modname="stap_c35cc695695a4d1cc021c5f060b93ddb__929"
stapio:init_stapio:369 init_stapio
stapio:init_ctl_channel:30 Opened .cmd (4)
stapio:stp_main_loop:567 in main loop
stapio:stp_main_loop:586 select_supported: 1
stapio:init_relayfs:244 initializing relayfs
stapio:init_relayfs:269 stapio:start_cmd:249 attempting to open trace0
blocking briefly
stapio:init_relayfs:269 attempting to open trace1
stapio:init_relayfs:281 ncpus=1, bulkmode = 0
stapio:init_relayfs:364 starting threads
stapio:stp_main_loop:738 systemtap_module_init() returned 0
stapio:stp_main_loop:744 detaching pid 930
stapio:start_cmd:268 execing target_cmd /home/elder/Application
test 
stapio:chld_proc:71 chld_proc 17 (Child exited)
Warning: child process exited with signal 11 (Segmentation fault)
INI
FIM 
stapio:stp_main_loop:729 got STP_REQUEST_EXIT
stapio:stp_main_loop:722 got STP_EXIT
stapio:cleanup_and_exit:454 detach=0
stapio:close_relayfs:380 closing
stapio:close_relayfs:399 done
stapio:cleanup_and_exit:471 closing control channel
stapio:close_ctl_channel:112 Closed ctl fd 4
stapio:cleanup_and_exit:479 
Renamed module to: stap_c35cc695695a4d1cc021c5f060b93ddb__929
stapio:cleanup_and_exit:486 removing stap_c35cc695695a4d1cc021c5f060b93ddb__929
staprun:parse_modpath:425 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko"
staprun:main:412 modpath="/lib/modules/3.7.2xen/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929.ko", modname="stap_c35cc695695a4d1cc021c5f060b93ddb__929"
staprun:init_staprun:314 init_staprun
staprun:remove_module:225 stap_c35cc695695a4d1cc021c5f060b93ddb__929
staprun:init_ctl_channel:72 Opened /sys/kernel/debug/systemtap/stap_c35cc695695a4d1cc021c5f060b93ddb__929/.cmd (4)
staprun:close_ctl_channel:112 Closed ctl fd 4
staprun:remove_module:250 removing module stap_c35cc695695a4d1cc021c5f060b93ddb__929
staprun:remove_module:262 Module stap_c35cc695695a4d1cc021c5f060b93ddb__929 removed.
Spawn waitpid result (0x100): 1
WARNING: /usr/local/bin/staprun exited with status: 1
Pass 5: run completed in 10usr/50sys/328real ms.
Pass 5: run failed.  Try again with another '--vp 00001' option.
Running rm -rf /tmp/stapBigAmY
Spawn waitpid result (0x0): 0
Removed temporary directory "/tmp/stapBigAmY"


My dmesg:
[ 9211.351238] stap_c35cc695695a4d1cc021c5f060b93ddb__929: systemtap: 2.1/0.153, base: ffffffffa062f000, memory: 55data/40text/2ctx/2058net/33alloc kb, probes: 3
[ 9211.354265] Application[930]: segfault at 0 ip 0000000000400596 sp 00007ffff8932d30 error 6 in Application[400000+1000]
Comment 5 Frank Ch. Eigler 2013-01-16 17:12:51 UTC
This sounds like a kernel uprobes bug, perhaps related to xen.
Comment 6 Frank Ch. Eigler 2020-04-24 17:10:05 UTC
current kernels seem to generally work