instrumenting kernel functions by name

[William Cohen]

During my quest to look for insteresting points to instrument it seems 
like I was rewriting it for each function I was trying out and 
recompiling the my test instrumentation quite a bit. I was looking at 
functions that were being called from a number of different places, so I 
am using the hash functions in the SystemTap Runtime library to keep 
statistics on a per caller basis. The attached 
kprobes_where_funct.tar.gz has the source code. You will need to point 
the build script in the executable to the appropriate place for the 
SystemTap runtime libraries. The build script is a cheat; is used to 
find out the address of the kallsym_lookup_name() even though it is not 
exported.

The instrumentation also makes use of the __print_symbol() function map 
the address back to the symbolic names.

Here is an example:
#  /sbin/insmod kprobe_where_funct.ko funct_name=io_schedule
#  /sbin/rmmod kprobe_where_funct

Output in /var/lib/messages
Mar 14 16:20:18 slingshot kernel: STP: kprobe unregistered
Mar 14 16:20:18 slingshot kernel: STP: io_schedule() called 22 times.
Mar 14 16:20:18 slingshot kernel: STP: io_schedule point break down.
Mar 14 16:20:18 slingshot kernel: STP: 0xc015f375, 22
Mar 14 16:20:18 slingshot kernel: STP: sync_buffer+0x3e/0x41

Another example:
# /sbin/insmod kprobe_where_funct.ko funct_name=__wake_up
# /sbin/rmmod kprobe_where_funct

Output in /var/log/messages:
Mar 14 16:24:06 slingshot kernel: STP: kprobe for __wake_up() registered 
at 0xc011d763
Mar 14 16:24:11 slingshot kernel: STP: kprobe unregistered
Mar 14 16:24:11 slingshot kernel: STP: __wake_up() called 295 times.
Mar 14 16:24:11 slingshot kernel: STP: __wake_up point break down.
Mar 14 16:24:11 slingshot kernel: STP: 0xc0121de6, 1
Mar 14 16:24:11 slingshot kernel: STP: release_console_sem+0xb5/0xb7
Mar 14 16:24:11 slingshot kernel: STP: 0xc02a205c, 11
Mar 14 16:24:11 slingshot kernel: STP: sock_def_readable+0x76/0x78
Mar 14 16:24:11 slingshot kernel: STP: 0xc02fd1c0, 1
Mar 14 16:24:11 slingshot kernel: STP: unix_dgram_recvmsg+0x9b/0x238
Mar 14 16:24:11 slingshot kernel: STP: 0xe0862a07, 34
Mar 14 16:24:11 slingshot kernel: STP: journal_stop+0x1e3/0x23b [jbd]
Mar 14 16:24:11 slingshot kernel: STP: 0xc0130465, 8
Mar 14 16:24:11 slingshot kernel: STP: __queue_work+0x45/0x54
Mar 14 16:24:12 slingshot kernel: STP: 0xc0130734, 8
Mar 14 16:24:12 slingshot kernel: STP: worker_thread+0x1d7/0x23a
Mar 14 16:24:12 slingshot kernel: STP: 0xc020e259, 148
Mar 14 16:24:12 slingshot kernel: STP: tty_ldisc_deref+0x67/0x82
Mar 14 16:24:12 slingshot kernel: STP: 0xc02129ad, 34
Mar 14 16:24:12 slingshot kernel: STP: n_tty_receive_buf+0xed/0xe09
Mar 14 16:24:12 slingshot kernel: STP: 0xe0866e08, 1
Mar 14 16:24:12 slingshot kernel: STP: __log_start_commit+0x31/0x3c [jbd]
Mar 14 16:24:12 slingshot kernel: STP: 0xe0866f8d, 1
Mar 14 16:24:12 slingshot kernel: STP: log_wait_commit+0x61/0x143 [jbd]
Mar 14 16:24:12 slingshot kernel: STP: 0xe086585f, 4
Mar 14 16:24:12 slingshot kernel: STP: 
__journal_remove_checkpoint+0x58/0x71 [jbd]
Mar 14 16:24:12 slingshot kernel: STP: 0xe08639ae, 2
Mar 14 16:24:12 slingshot kernel: STP: 
journal_commit_transaction+0x23a/0x10ec [jbd]
Mar 14 16:24:12 slingshot kernel: STP: 0xc020e720, 22
Mar 14 16:24:12 slingshot kernel: STP: tty_wakeup+0x6b/0x7d
Mar 14 16:24:12 slingshot kernel: STP: 0xc0134539, 6
Mar 14 16:24:12 slingshot kernel: STP: __wake_up_bit+0x33/0x38
Mar 14 16:24:12 slingshot kernel: STP: 0xe08644b2, 2
Mar 14 16:24:12 slingshot kernel: STP: 
journal_commit_transaction+0xd3e/0x10ec [jbd]
Mar 14 16:24:12 slingshot kernel: STP: 0xe0866789, 2
Mar 14 16:24:12 slingshot kernel: STP: kjournald+0xf3/0x231 [jbd]
Mar 14 16:24:12 slingshot kernel: STP: 0xc0169899, 2
Mar 14 16:24:12 slingshot kernel: STP: pipe_release+0x6d/0xbd
Mar 14 16:24:12 slingshot kernel: STP: 0xc02fb442, 8
Mar 14 16:24:12 slingshot kernel: STP: unix_release_sock+0xbc/0x25c


I would appreciate comments on this code.

-Will
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kprobe_where_funct.tar.gz
Type: application/x-tar
Size: 1714 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/systemtap/attachments/20050314/731893e3/attachment.tar>