[Bug runtime/9989] New: utrace or task-finder crash in upstream utrace git tree

["fche at redhat dot com" <sourceware-bugzilla at sourceware dot org>]

kernel 2.6.29-rc8-00313-g13a5838, built from roland's utrace git tree,
crashes during the git systemtap installcheck testsuite.  repeatable.

stap_8e70ec617e7a5e5b56ae7b4523e18922_19315: systemtap: 0.9/0.139, base:
ffffffffa01fe000, memory: 2
0604+61188+13136+14000 data+text+ctx+net, probes: 30
stap_593b559747144f0a15e5ecabd4cb570d_769: systemtap: 0.9/0.139, base:
ffffffffa0089000, memory: 116
36+10988+656+14000 data+text+ctx+net, probes: 2
stap_37623d5ee886870c5fd440c1850af20a_2122: systemtap: 0.9/0.139, base:
ffffffffa0214000, memory: 51
084+19012+8496+14000 data+text+ctx+net, probes: 2
stap_5c5baabdd4d19dc1f906b4bbf8c59c7e_578: systemtap: 0.9/0.139, base:
ffffffffa0228000, memory: 224
05+18795+656+14000 data+text+ctx+net, probes: 3
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<(null)>] (null)
PGD 6ed8b067 PUD 7b499067 PMD 0 
Oops: 0010 [#1] SMP 
last sysfs file: /sys/devices/pci0000:00/0000:00:03.0/local_cpus
CPU 0 
Modules linked in: uprobes nfs lockd nfs_acl auth_rpcgss iptable_nat nf_nat sco
bridge stp bnep l2cap bluetooth sunrpc ip6t_REJECT nf_conntrack_ipv6
ip6table_filter ip6_tables ipv6 dm_multipath uinput ppdev floppy 8139cp pcspkr
8139too mii i2c_piix4 i2c_core parport_pc parport ata_generic pata_acpi [last
unloaded: stap_5c5baabdd4d19dc1f906b4bbf8c59c7e_578]
Pid: 3344, comm: expect Not tainted 2.6.29-rc8-00313-g13a5838 #56 
RIP: 0010:[<0000000000000000>]  [<(null)>] (null)
RSP: 0018:ffff88006eda3e48  EFLAGS: 00010282
RAX: ffffffffa022fdd0 RBX: ffff880049c27810 RCX: 0000000001200011
RDX: ffff8800769b5c80 RSI: ffff880049c27810 RDI: 0000000000000005
RBP: ffff88006eda3ea0 R08: ffff8800391c8000 R09: 0000000000000000
R10: ffff8800391c8000 R11: ffff88006eda3e10 R12: ffff8800769b5c80
R13: ffff8800769b6248 R14: ffff88006eda3e60 R15: ffff8800391c8000
FS:  00007fb962a226f0(0000) GS:ffffffff81776000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 000000007154b000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process expect (pid: 3344, threadinfo ffff88006eda2000, task ffff8800769b5c80)
Stack:
 ffffffff8107f575 0000000001200011 ffff8800769b6250 0000000000000005
 ffff880000010000 000000007b492840 0000000000000000 ffff8800391c8000
 0000000000000000 0000000001200011 0000000000006a0a ffff88006eda3f30
Call Trace:
 [<ffffffff8107f575>] ? utrace_report_clone+0xa4/0x128
 [<ffffffff8104824e>] do_fork+0x215/0x35b
 [<ffffffff8135f54d>] ? _spin_lock+0x9/0xc
 [<ffffffff810d6f14>] ? path_put+0x1d/0x21
 [<ffffffff8100f649>] sys_clone+0x23/0x25
 [<ffffffff810117e3>] stub_clone+0x13/0x20
 [<ffffffff8101133a>] ? system_call_fastpath+0x16/0x1b
Code:  Bad RIP value.
RIP  [<(null)>] (null)
 RSP <ffff88006eda3e48>
CR2: 0000000000000000
---[ end trace 3b446d259688f9aa ]---
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<(null)>] (null)
PGD 7a000067 PUD 7b12e067 PMD 0 

... and gets worse from there.

In another instance,

stap_092674bdf95c7926476797ef8f6289a7_2122: systemtap: 0.9/0.139, base:
ffffffffa0421000, memory: 51076+19020+8496+14000 data+text+ctx+net, probes: 2
stap_6fc07325b5ca3ccfad563565b1f93382_578: systemtap: 0.9/0.139, base:
ffffffffa0435000, memory: 22413+18787+656+14000 data+text+ctx+net, probes: 3
stap_89dd075586e5fc9db6c6b0caeb359cc6_590: systemtap: 0.9/0.139, base:
ffffffffa0442000, memory: 22429+18787+656+14000 data+text+ctx+net, probes: 3
BUG: unable to handle kernel paging request at 000000000000ea60
IP: [<000000000000ea60>] 0xea60
PGD 7ad64067 PUD 754f3067 PMD 0 
Oops: 0010 [#1] SMP 
last sysfs file: /sys/module/uprobes/sections/__ksymtab_strings
CPU 1 
Modules linked in: stap_89dd075586e5fc9db6c6b0caeb359cc6_590 uprobes nfs lockd
nfs_acl auth_rpcgss iptable_nat nf_nat sco bridge stp bnep l2cap bluetooth
sunrpc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6
dm_multipath uinput ppdev 8139cp 8139too floppy i2c_piix4 mii parport_pc pcspkr
i2c_core parport ata_generic pata_acpi [last unloaded:
stap_6fc07325b5ca3ccfad563565b1f93382_578]
Pid: 2864, comm: rsyslogd Not tainted 2.6.29-rc8-00313-g13a5838 #56 
RIP: 0010:[<000000000000ea60>]  [<000000000000ea60>] 0xea60
RSP: 0018:ffff88007b581ea0  EFLAGS: 00010282
RAX: ffffffffa043cdd0 RBX: ffff8800471e0240 RCX: 00000000ffffffff
RDX: 0000000000000000 RSI: ffff88007013c560 RDI: ffff8800471e0240
RBP: ffff88007b581ef8 R08: ffff88007b581e88 R09: ffff88007013c560
R10: ffff88007b581ef8 R11: ffff88007b581ef8 R12: ffff88007013cb28
R13: ffff88007013c560 R14: ffff88007013cb50 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff88007cc35580(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000000ea60 CR3: 000000007b9b1000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process rsyslogd (pid: 2864, threadinfo ffff88007b580000, task ffff88007013c560)
Stack:
 ffffffff8107f997 ffffffff7013c560 ffff88007013cb30 0000000000000005
 ffffffff00010000 ffff88007b581ee8 ffff88007013c560 00000000ffffffff
 0000000000000000 0000003759c17280 0000000000000000 ffff88007b581f28
Call Trace:
 [<ffffffff8107f997>] ? utrace_report_death+0xb9/0x12a
 [<ffffffff8104c430>] exit_notify+0x157/0x179
 [<ffffffff8104c6dd>] do_exit+0x28b/0x31b
 [<ffffffff8104c844>] complete_and_exit+0x0/0x20
 [<ffffffff8101133a>] system_call_fastpath+0x16/0x1b
Code:  Bad RIP value.
RIP  [<000000000000ea60>] 0xea60
 RSP <ffff88007b581ea0>
CR2: 000000000000ea60
---[ end trace ed3c61cc990102ec ]---
Fixing recursive fault but reboot is needed!
stap_c18d89ac41064f6d7bbb5323d710562e_631: systemtap: 0.9/0.139, base:
ffffffffa044f000, memory: 22469+18939+656+14000 data+text+ctx+net, probes: 3

note that RAX here points to the second-last systemtap module, as if
utrace cleanup was not properly completed.

The systemtap test cases being run/attempted are a bit tricky to identify
from the hash code / leftovers in the cache, but I believe they are the
systemtap.base/utrace_p* series.

I have not yet eliminated recent systemtap changes for this regression,
but at this early stage it seems equally possible that this is a problem
with this version of utrace (thus cc: roland).

-- 
           Summary: utrace or task-finder crash in upstream utrace git tree
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: runtime
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: fche at redhat dot com
                CC: roland at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=9989

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.