Concurrent userspace probes targeting same location

Wade Farnsworth wade_farnsworth@mentor.com
Tue Jul 3 20:07:00 GMT 2012


Greetings,

I'm seeing some odd behavior when multiple userspace return probes are 
running attached to the same target location.

First, the return value of uaddr() of one probe points to a location 
inside of the second.  The second probe's uaddr() behaves as expected.

Second, it appears that occasionally a probe will get executed twice for 
a specific function return.

I have observed these problems on x86/Fedora 16, and ARM/Yocto.

Here are some sample probes that exhibit the problems:

probe process("/bin/ls").library("libc.so.6").function("malloc").return
{
     if (usymname(uaddr()) == "__libc_malloc")
         next;
     printf("malloc probe 1 -- ptr: 0x%x, return addr 0x%x\n", $return, 
uaddr());
}

probe process("/bin/ls").library("libc.so.6").function("malloc").return
{
     if (usymname(uaddr()) == "__libc_malloc")
         next;
     printf("malloc probe 2 -- ptr: 0x%x, return addr 0x%x\n", $return, 
uaddr());
}

And here is a snippet of the output when these are run simultaneously on 
Fedora 16:

malloc probe 2 -- ptr: 0x9b12008, return addr 0xbfcbe010
malloc probe 2 -- ptr: 0x9b12008, return addr 0xbfcbe010
malloc probe 1 -- ptr: 0x9b12008, return addr 0x446f1901
malloc probe 2 -- ptr: 0x9b12020, return addr 0xbfcbe010
malloc probe 1 -- ptr: 0x9b12020, return addr 0x4469f1fc
malloc probe 2 -- ptr: 0x9b12030, return addr 0xbfcbe010
malloc probe 1 -- ptr: 0x9b12030, return addr 0x44698da2
malloc probe 2 -- ptr: 0x9b12020, return addr 0xbfcbe010
...

The lines prefixed with "malloc probe 2" consistently point into the 
kernel module space (0xbfcbe010), instead of into userspace.  Also, the 
first two lines demonstrate the double execution problem (note the 
identical values of $return).

Has anyone experienced anything similar?  Is this even a valid use case?

Thanks,

Wade



More information about the Systemtap mailing list