Looking for recommendation for using SystemTap

[David Wilder]

Tony Reix wrote:

>Hi,
>
>I'm having several Oopss while running tests of an application which
>has:
> - one patch applied to the kernel
> - one kernel module
>The analysis of the Oopss clearly show that "someone" writes strings
>(like "ata" or "ejbo") randomly in memory and destroys links in
>structures, like vmlilst used by get_vmalloc_info in fs/proc/mmu.c or
>ulp->proc_list used by loop_undo in ipc/sems.c .
>Maybe my code is the culprit, or not.
>
>Do you think SystemTap can help me finding the culprit ?
>If yes, do you have recommendations and proposals about how to use
>SystemTap for that goal ?
>Can you point me to documentations providing the basic for using
>SystemTap in real ?
>
>Thanks,
>
>Tony
>
>  
>
A combination of systemtap and kdump may help.
My idea is to write a tap script that detects the corruption by 
searching for the corruption pattern.  If you know the section of memory 
or the list that is often corrupted you could limit the search area to 
speed up the probe.  Then place the probe (maybe a return probe would 
work better) at suspected areas of the code.  It may take several trys.  
The idea is to get closer to the point where the corruption occurred.  
It is possible to do probe all kernel functions using a *.  But that may 
be overkill.  Your tap script could call panic() to trigger a crash dump 
when the corruption is detected.  Searching the active stacks in the 
dump may get you closer to the bad code.  You can also look for patterns 
between several dumps, like a particular task or subsystem always on 
CPU.  Using that info move your probe point around.

I hope this helps.

-- 
David Wilder
IBM Linux Technology Center
Beaverton, Oregon, USA 
dwilder@us.ibm.com
(503)578-3789