Re: Kernel panics during execution of timeouts.stp example when booted with kgdboc=kms,kbd

[fche at redhat dot com (Frank Ch. Eigler)]

<dmitry.krivenok@emc.com> writes:

> I'm new to Systemtap, but already found it very useful and even
> solved some real problems using it.

Great!

> The problem is that kernel panics if you run it with
> "kgdboc=kms,kbd" parameter and then run standard timeout.stp example
> as follows: [...]

I can reproduce this crash even on plain kvm, in this case running
systemtap: 1.6/0.152, 2.6.41.1-1.fc15.x86_64, 2 VCPU.

Entering kdb (current=0xffff88007a4c1730, pid 1667) on processor 0 Oops: (null)
due to oops @ 0xffffffff81076c33
CPU 0 <d>Modules linked in: stap_7cd19ef8989e5e76ef083a66b68e4d39__1667 iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 netconsole configfs nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ppdev 8139too parport_pc parport 8139cp i2c_piix4 i2c_core mii ipv6 [last unloaded: scsi_wait_scan]
<c>
<d>Pid: 1667, comm: stapio Not tainted 2.6.41.1-1.fc15.x86_64 #1<c> Bochs Bochs<c>
<d>RIP: 0010:[<ffffffff81076c33>]  [<ffffffff81076c33>] sys_nanosleep+0x1/0x6a
<d>RSP: 0018:ffff880079367f80  EFLAGS: 00000297
<d>RAX: 0000000000000023 RBX: ffffffffffffffff RCX: 00002ba8344af9d0
<d>RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00007fff249c12a0
<d>RBP: 00002ba82cda7010 R08: 00002ba8344af700 R09: 00002ba8344af700
<d>R10: 00002ba8344af9d0 R11: 0000000000000293 R12: 0000000000008002
<d>R13: 00002ba82cdad178 R14: 00002ba82cda7008 R15: 000000000000000b
<d>FS:  00002ba82d585120(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
<d>CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<d>CR2: 00002ba82d2c3920 CR3: 0000000037635000 CR4: 00000000000006f0
<d>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<d>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process stapio (pid: 1667, threadinfo ffff880079366000, task ffff88007a4c1730)
<0>Stack:
<c> ffffffff814a3102<c> 0000000000000293<c> 00002ba8344af9d0<c> 00002ba8344af700<c>
<c> 00002ba8344af700<c> 0000000000000023<c> 00002ba82cfdd1e5<c> 0000000000000000<c>
<c> 0000000000000000<c> 00007fff249c12a0<c> 0000000000000023<c> 00002ba82d2956ad<c>
<0>Call Trace:


Without the kgdboc=kms,kbd boot options, the test case works fine.
This leads me to believe that there is a conflict between the kernel's
kretprobes and kgdb systems, and likely not a bug in systemtap proper.
(I haven't been able to trigger the problem with a couple of 'perf
probe' attempts though.)  However, checking the hack associated with
http://sourceware.org/PR13193, disabling kprobes optimization makes
this test case work for me.  Please try

# echo 0 > /proc/sys/debug/kprobes-optimization

and rerun your tests.  The next release of systemtap will probably
include code to do the same thing. :-(

- FChE