This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.

Index Nav:	[Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav:	[Date Prev] [Date Next]	[Thread Prev] [Thread Next]
Other format:	[Raw text]

Re: Script to measure resource usage based on process arguments

From: Mike Mason <mmlnx at us dot ibm dot com>
To: Mike Mason <mmlnx at us dot ibm dot com>
Cc: systemtap at sources dot redhat dot com
Date: Mon, 08 Jan 2007 17:27:07 -0800
Subject: Re: Script to measure resource usage based on process arguments
References: <45A2E7D0.6030809@us.ibm.com>

Forgot to mention one important thing... I've seen this script crash the system a few times. The sequence of events that cause the crash are a bit strange. I start the script, then start a kernel build. The script and build both run to completion. Then I start the script again and it crashes. I've attached an example of the console messages. Do these messages give anybody a clue as to what's happening?

BTW, I see the crash when I run the script in an xterm in Xvnc. I don't see the crash when I run the script in a telnet session. Might be irrelevant, but I thought I'd mention it.

Mike

Console messages
================

Error creating systemtap /proc entries. Unable to handle kernel paging request at ffffffff8832e1dd RIP: [<ffffffff8832e1dd>] PGD 203027 PUD 205027 PMD 396e6067 PTE 0 Oops: 0010 [1] SMP last sysfs file: /module/scsi_mod/sections/.text CPU 0 Modules linked in: ipv6 autofs4 hidp rfcomm l2cap bluetooth sunrpc dm_mirror dm_mod video sbs i2c_ec button battery asus_acpi ac lp parport_pc parport snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss sg e1000 snd_mixer_oss snd_pcm serio_raw snd_timer ehci_hcd snd uhci_hcd ide_cd i2c_i801 shpchp soundcore snd_page_alloc cdrom i2c_core pcspkr ext3 jbd ahci libata sd_mod scsi_mod Pid: 2467, comm: Xvnc Not tainted 2.6.18-1.2257.fc5 #1 RIP: 0010:[<ffffffff8832e1dd>] [<ffffffff8832e1dd>] RSP: 0000:ffffffff80632f18 EFLAGS: 00010246 RAX: ffff81002d773fd8 RBX: 0000000000000000 RCX: ffff81001aae2378 RDX: 0000000000000000 RSI: 10369bb9244ede90 RDI: 0000000000000000 RBP: ffffffff80671b00 R08: 0000000000b31860 R09: 000000000000000a R10: 00002aaaaadafa18 R11: 0000000000000004 R12: 0000000000000100 R13: ffffffff8832e1dd R14: 0000000000000027 R15: 0000000000000105 FS: 00002aaaaaacd140(0000) GS:ffffffff805d8000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffff8832e1dd CR3: 000000002e048000 CR4: 00000000000006e0 Process Xvnc (pid: 2467, threadinfo ffff81002d772000, task ffff81003848b810) Stack: ffffffff80290ec5 ffffffff80632f20 ffffffff80632f20 ffffffff80287132 0000000000000001 ffffffff805de110 000000000000000a 0000000000000000 ffffffff80211cff ffff81002d773f58 0000000000000046 ffffffff80632f98 Call Trace: Inexact backtrace: <IRQ> [<ffffffff80290ec5>] run_timer_softirq+0x13b/0x1be [<ffffffff80287132>] scheduler_tick+0x31c/0x345 [<ffffffff80211cff>] __do_softirq+0x5e/0xd5 [<ffffffff8025d9f0>] call_softirq+0x1c/0x28 [<ffffffff8026aab8>] do_softirq+0x2c/0x99 [<ffffffff8025d382>] apic_timer_interrupt+0x66/0x6c <EOI>

Code:  Bad RIP value.
RIP  [<ffffffff8832e1dd>]
RSP <ffffffff80632f18>
CR2: ffffffff8832e1dd
<1>Unable to handle kernel paging request<3>BUG: sleeping function called from invalid context at kernel/rwsem.c:20
in_atomic():1, irqs_disabled():1

Call Trace: [<ffffffff80269935>] show_trace+0x34/0x47 [<ffffffff8026995a>] dump_stack+0x12/0x17 [<ffffffff8029bb57>] down_read+0x15/0x23 [<ffffffff8029414e>] blocking_notifier_call_chain+0x13/0x36 [<ffffffff80214e81>] do_exit+0x20/0x928 [<ffffffff80265023>] do_page_fault+0x79a/0x815 [<ffffffff8025d4dd>] error_exit+0x0/0x84 DWARF2 unwinder stuck at error_exit+0x0/0x84 Leftover inexact backtrace: <IRQ> [<ffffffff80290ec5>] run_timer_softirq+0x13b/0x1be [<ffffffff80287132>] scheduler_tick+0x31c/0x345 [<ffffffff80211cff>] __do_softirq+0x5e/0xd5 [<ffffffff8025d9f0>] call_softirq+0x1c/0x28 [<ffffffff8026aab8>] do_softirq+0x2c/0x99 [<ffffffff8025d382>] apic_timer_interrupt+0x66/0x6c <EOI> Kernel panic - not syncing: Aiee, killing interrupt handler! at ffffffff8832e1dd RIP: [<ffffffff8832e1dd>] PGD 203027 PUD 205027 PMD 396e6067 PTE 0 Oops: 0010 [2] SMP last sysfs file: /module/scsi_mod/sections/.text CPU 1 Modules linked in: ipv6 autofs4 hidp rfcomm l2cap bluetooth sunrpc dm_mirror dm_mod video sbs i2c_ec button battery asus_acpi ac lp parport_pc parport snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss sg e1000 snd_mixer_oss snd_pcm serio_raw snd_timer ehci_hcd snd uhci_hcd ide_cd i2c_i801 shpchp soundcore snd_page_alloc cdrom i2c_core pcspkr ext3 jbd ahci libata sd_mod scsi_mod Pid: 24434, comm: insmod Not tainted 2.6.18-1.2257.fc5 #1 RIP: 0010:[<ffffffff8832e1dd>] [<ffffffff8832e1dd>] RSP: 0018:ffff81003fe0bf18 EFLAGS: 00010246 RAX: ffff810004d7bfd8 RBX: 0000000000000000 RCX: ffff81001aae2678 RDX: 0000000000000001 RSI: 10369bb9244ede90 RDI: 0000000000000000 RBP: ffff810037ebc000 R08: 0000000000b31860 R09: 000000000000000a R10: ffff810001182d90 R11: 0000000000000246 R12: 0000000000000100 R13: ffffffff8832e1dd R14: 0000000000000100 R15: 0000000000000003 FS: 0000000000000000(0000) GS:ffff810037e999c0(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffff8832e1dd CR3: 0000000006a2b000 CR4: 00000000000006e0 Process insmod (pid: 24434, threadinfo ffff810004d7a000, task ffff810039312080) Stack: ffffffff80290ec5 ffff81003fe0bf20 ffff81003fe0bf20 ffffffff80287132 0000000000000011 ffffffff805de110 000000000000000a 0000000000000001 ffffffff80211cff ffff810004d7bde8 0000000000000046 ffff81003fe0bf98 Call Trace: Inexact backtrace: <IRQ> [<ffffffff80290ec5>] run_timer_softirq+0x13b/0x1be [<ffffffff80287132>] scheduler_tick+0x31c/0x345 [<ffffffff80211cff>] __do_softirq+0x5e/0xd5 [<ffffffff8025d9f0>] call_softirq+0x1c/0x28 [<ffffffff8026aab8>] do_softirq+0x2c/0x99 [<ffffffff8025d382>] apic_timer_interrupt+0x66/0x6c <EOI> [<ffffffff80262eac>] _spin_unlock_irq+0xb/0xc [<ffffffff8020cf32>] __pagevec_lru_add_active+0xb9/0xdc [<ffffffff80239e82>] exit_mmap+0x22/0xee [<ffffffff8023c058>] mmput+0x41/0x96 [<ffffffff802150f4>] do_exit+0x293/0x928 [<ffffffff8024816e>] cpuset_exit+0x0/0x6c [<ffffffff8025c80e>] system_call+0x7e/0x83


Code:  Bad RIP value.
RIP  [<ffffffff8832e1dd>]
RSP <ffff81003fe0bf18>
CR2: ffffffff8832e1dd
BUG: warning at drivers/char/vt.c:3359/do_unblank_screen() (Not tainted)

Call Trace:
[<ffffffff80269935>] show_trace+0x34/0x47
[<ffffffff8026995a>] dump_stack+0x12/0x17
[<ffffffff8038b5a6>] do_unblank_screen+0x67/0x142
[<ffffffff8027c523>] bust_spinlocks+0x1c/0x46
[<ffffffff802634ce>] oops_end+0x15/0x53
[<ffffffff80264fd3>] do_page_fault+0x74a/0x815
[<ffffffff8025d4dd>] error_exit+0x0/0x84
DWARF2 unwinder stuck at error_exit+0x0/0x84
Leftover inexact backtrace:
<IRQ>  [<ffffffff80290ec5>] run_timer_softirq+0x13b/0x1be
[<ffffffff80287132>] scheduler_tick+0x31c/0x345
[<ffffffff80211cff>] __do_softirq+0x5e/0xd5
[<ffffffff8025d9f0>] call_softirq+0x1c/0x28
[<ffffffff8026aab8>] do_softirq+0x2c/0x99
[<ffffffff8025d382>] apic_timer_interrupt+0x66/0x6c
<EOI>  [<ffffffff80262eac>] _spin_unlock_irq+0xb/0xc
[<ffffffff8020cf32>] __pagevec_lru_add_active+0xb9/0xdc
[<ffffffff80239e82>] exit_mmap+0x22/0xee
[<ffffffff8023c058>] mmput+0x41/0x96
[<ffffffff802150f4>] do_exit+0x293/0x928
[<ffffffff8024816e>] cpuset_exit+0x0/0x6c
[<ffffffff8025c80e>] system_call+0x7e/0x83

Follow-Ups:
- Re: Script to measure resource usage based on process arguments
  - From: Frank Ch. Eigler

References:
- Script to measure resource usage based on process arguments
  - From: Mike Mason

Index Nav:	[Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav:	[Date Prev] [Date Next]	[Thread Prev] [Thread Next]