Bug 27260 - uprobes is unstable on centos6 x86_64 when running the same script repeatedly against the same target process
Summary: uprobes is unstable on centos6 x86_64 when running the same script repeatedly...
Status: RESOLVED WONTFIX
Alias: None
Product: systemtap
Classification: Unclassified
Component: uprobes (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-01-28 02:07 UTC by agentzh
Modified: 2023-10-30 15:26 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description agentzh 2021-01-28 02:07:49 UTC
We noted that use of uprobes on centos6 x86_64 is very unstable even with the simplest script and the simplest target C program. Below is a minimal example that can reliably reproduce the issue:

First of all, we create a simple C program as the target:


```
#include <stdio.h>

void foo(int i) {
    printf("hello %d\n", i);
}

int main(void) {
    int i = 0;
    while (1) {
        foo(i++);
    }
    return 0;
}
```

And then compile this C program without inlining:

```
gcc -O -fno-inline -g a.c
```

Run the target program forever:

```
./a.out > /dev/null
```

Finally, run a stap oneliner in a bash loop:

```
$ while true; do /opt/stap/bin/stap -d ~/a.out -e 'probe process("/home/agentzh/a.out").function("foo") { println("Hit"); exit() }'; done
Hit
Hit
Hit
Hit
Hit
Hit
Hit
Hit
Hit
```

This loop will quickly hang sooner or later. Some times after 3 Hits, some times after about 10.

And in the terminal running the ./a.out, we can see the target is already killed:

```
[agentzh@centos6-pkg ~]$ ./a.out > /dev/null
Trace/breakpoint trap
[agentzh@centos6-pkg ~]$
```

I'm using the latest stap master (commit a790dc64a9d8d9):

```
[agentzh@centos6-pkg systemtap]$ /opt/stap/bin/stap -V
Systemtap translator/driver (version 4.5/0.177, commit release-4.4-74-ga790dc64a9d8 + changes)
Copyright (C) 2005-2020 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.32 ... 5.10.0-rc
enabled features: JAVA LIBXML2 READLINE
```

Using the following centos6 stock kernel:

```
[agentzh@centos6-pkg systemtap]$ uname -r
2.6.32-754.18.2.el6.x86_64
```

Tested all the centos 6 kernels in the official repo. Same thing.
Comment 1 William Cohen 2023-10-30 15:26:22 UTC
RHEL6 is out of support November 2020 (https://access.redhat.com/discussions/4768501) and support for 2.6.32 was recently removed from systemtap (pr30434), so this is being closed.