KGTP (Linux Kernel debugger and tracer) 20110424 release

[Hui Zhu <teawater at gmail dot com>]

KGTP is a realtime and lightweight Linux Kernel GDB debugger and
tracer that use Kprobe.

It make Linux Kernel supply a GDB remote debug interface. Then GDB in
current machine or remote machine(see "Make GDB connect to gtp") can
debug Linux through GDB tracepoint without stop the Linux Kernel. And
even if the board doesn't have GDB on it and doesn't have
interface for remote debug. It can debug the Linux Kernel use offline
debug (See "Offline debug").

It support X86-32, X86-64, MIPS and ARM.
http://www.tudou.com/programs/view/_p6VTgxmCRA/ This is a video(in
Chinese) to show how to use it.

Now, KGTP 20110424 release.
You can get the package for it from
http://kgtp.googlecode.com/files/kgtp_20110424.tar.bz2
or
svn co https://kgtp.googlecode.com/svn/tags/20110424

The main change of this verion is add a special trace state variables
$dump_stack, "collect" it will let Linux Kernel output stack dump
directly.
Following example let Linux Kernel show the stack dump of vfs_readdir:
trace vfs_readdir
  commands
    collect $dump_stack
  end
Then your kernel will printk like:

[22779.208064] gtp 1:Pid: 441, comm: python Not tainted 2.6.39-rc3+ #46
[22779.208068] Call Trace:
[22779.208072]  [<fe653cca>] gtp_get_var+0x4a/0xa0 [gtp]
[22779.208076]  [<fe653d79>] gtp_collect_var+0x59/0xa0 [gtp]
[22779.208080]  [<fe655974>] gtp_action_x+0x1bb4/0x1dc0 [gtp]
[22779.208084]  [<c05b6408>] ? _raw_spin_unlock+0x18/0x40
[22779.208088]  [<c023f152>] ? __find_get_block_slow+0xd2/0x160
[22779.208091]  [<c01a8c56>] ? delayacct_end+0x96/0xb0
[22779.208100]  [<c023f404>] ? __find_get_block+0x84/0x1d0
[22779.208103]  [<c05b6408>] ? _raw_spin_unlock+0x18/0x40
[22779.208106]  [<c02e0838>] ? find_revoke_record+0xa8/0xc0
[22779.208109]  [<c02e0c45>] ? jbd2_journal_cancel_revoke+0xd5/0xe0
[22779.208112]  [<c02db51f>] ? __jbd2_journal_temp_unlink_buffer+0x2f/0x110
[22779.208115]  [<fe655c4c>] gtp_kp_pre_handler+0xcc/0x1c0 [gtp]
[22779.208118]  [<c05b8a88>] kprobe_exceptions_notify+0x3d8/0x440
[22779.208121]  [<c05b7d54>] ? hw_breakpoint_exceptions_notify+0x14/0x180
[22779.208124]  [<c05b95eb>] ? sub_preempt_count+0x7b/0xb0
[22779.208126]  [<c0227ac5>] ? vfs_readdir+0x15/0xb0
[22779.208128]  [<c0227ac4>] ? vfs_readdir+0x14/0xb0
[22779.208131]  [<c05b9743>] notifier_call_chain+0x43/0x60
[22779.208134]  [<c05b9798>] __atomic_notifier_call_chain+0x38/0x50
[22779.208137]  [<c05b97cf>] atomic_notifier_call_chain+0x1f/0x30
[22779.208140]  [<c05b980d>] notify_die+0x2d/0x30
[22779.208142]  [<c05b71c5>] do_int3+0x35/0xa0

Add a special trace state variables $clock.  Access it in tracepoint
condition and action will get the return of local_clock() that return
the timestamp in nanoseconds.

And according to Steven Rostedt's mail, move the interface from proc
to debug fs.  So, access kgtp need change to /sys/kernel/debug/gtp.

The other change is:
Make frame align better.
Change sp_checked to type 0xff, Change stack of x from local to static.
Fix the bug of $printk_tmp.
Make getgtprsp.pl support special trace state variables.

You can read the doc gtp.txt to get how to use kgtp.

And according to the comments of Christoph.  I make a patch for Linux
Kernel and make it looks OK with checkpatch.pl.

This patch is too big for this maillist, So I just post the link:
https://lkml.org/lkml/2011/4/24/45