]>
Commit | Line | Data |
---|---|---|
210ff7bf | 1 | SYSTEMTAP EXAMPLES INDEX BY KEYWORD |
09a98ae0 | 2 | (see also index.txt) |
210ff7bf | 3 | |
8b88e771 FCE |
4 | = AUTOFS = |
5 | ||
6 | network/autofs4.stp - Watch autofs4 operations | |
7 | keywords: network autofs nfs | |
8 | ||
9 | Trace key autofs4 operations such as mounting or unmounting remote | |
10 | filesystems. | |
11 | ||
12 | ||
210ff7bf FCE |
13 | = BACKTRACE = |
14 | ||
1cc8a4c7 WC |
15 | interrupt/scf.stp - Tally Backtraces for Inter-Processor Interrupt (IPI) |
16 | keywords: interrupt backtrace | |
17 | ||
18 | The Linux kernel function smp_call_function causes expensive | |
19 | inter-processor interrupts (IPIs). The scf.stp script tallies the | |
20 | processes and backtraces causing the interprocessor interrupts to | |
21 | identify the cause of the expensive IPI. On exit the script prints | |
22 | the tallies in descending frequency. | |
23 | ||
24 | ||
210ff7bf | 25 | io/io_submit.stp - Tally Reschedule Reason During AIO io_submit Call |
09a98ae0 | 26 | keywords: io backtrace |
210ff7bf FCE |
27 | |
28 | When a reschedule occurs during an AIO io_submit call, accumulate the | |
29 | traceback in a histogram. When the script exits prints out a sorted | |
30 | list from most common to least common backtrace. | |
31 | ||
32 | ||
cae71dd3 | 33 | process/sleepingBeauties.stp - Generating Backtraces of Threads Waiting for IO Operations |
09a98ae0 | 34 | keywords: io scheduler backtrace |
cae71dd3 FCE |
35 | |
36 | The script monitors the time that threads spend waiting for IO | |
37 | operations (in "D" state) in the wait_for_completion function. If a | |
38 | thread spends over 10ms, its name and backtrace is printed, and later | |
39 | so is the total delay. | |
40 | ||
41 | ||
0e4901b0 WC |
42 | = BUFFER = |
43 | ||
44 | network/dropwatch.stp - Watch Where Socket Buffers are Freed in the Kernel | |
45 | keywords: network tracepoint buffer free | |
46 | ||
47 | Every five seconds the dropwatch.stp script lists the number of | |
48 | socket buffers freed at locations in the kernel. | |
49 | ||
50 | ||
905728a0 WC |
51 | network/sk_stream_wait_memory.stp - Track Start and Stop of Processes Due to Network Buffer Space |
52 | keywords: network tcp buffer memory | |
53 | ||
54 | The sk_stream-wait_memory.stp prints a time stamp, executable, and | |
55 | pid each time a process blocks due to the send buffer being full. A | |
56 | similar entry is printed each time a process continues because there | |
57 | is room in the buffer. | |
58 | ||
59 | ||
210ff7bf FCE |
60 | = CALLGRAPH = |
61 | ||
aa15b9f0 | 62 | general/para-callgraph.stp - Callgraph tracing with arguments |
09a98ae0 | 63 | keywords: trace callgraph |
210ff7bf | 64 | |
aa15b9f0 FCE |
65 | Print a timed per-thread callgraph, complete with function parameters |
66 | and return values. The first parameter names the function probe | |
67 | points to trace. The optional second parameter names the probe | |
68 | points for trigger functions, which acts to enable tracing for only | |
69 | those functions that occur while the current thread is nested within | |
70 | the trigger. | |
210ff7bf FCE |
71 | |
72 | ||
73 | = CPU = | |
74 | ||
75 | general/graphs.stp - Graphing Disk and CPU Utilization | |
09a98ae0 | 76 | keywords: disk cpu use graph |
210ff7bf FCE |
77 | |
78 | The script tracks the disk and CPU utilization. The resulting output | |
79 | of the script can be piped into gnuplot to generate a graph of disk | |
80 | and CPU USE. | |
81 | ||
82 | ||
83 | = DISK = | |
84 | ||
85 | general/graphs.stp - Graphing Disk and CPU Utilization | |
09a98ae0 | 86 | keywords: disk cpu use graph |
210ff7bf FCE |
87 | |
88 | The script tracks the disk and CPU utilization. The resulting output | |
89 | of the script can be piped into gnuplot to generate a graph of disk | |
90 | and CPU USE. | |
91 | ||
92 | ||
93 | io/disktop.stp - Summarize Disk Read/Write Traffic | |
09a98ae0 | 94 | keywords: disk |
210ff7bf FCE |
95 | |
96 | Get the status of reading/writing disk every 5 seconds, output top | |
97 | ten entries during that period. | |
98 | ||
99 | ||
0e0b566a WC |
100 | = FORMAT = |
101 | ||
102 | general/ansi_colors.stp - Color Table for ansi_set_color2() and ansi_set_color3() | |
103 | keywords: format | |
104 | ||
105 | The script prints a table showing the available color combinations | |
106 | for the ansi_set_color2() and ans_set_color3() functions in the | |
107 | ansi.stp tapset. | |
108 | ||
109 | ||
110 | general/ansi_colors2.stp - Show Attribues in Table for ansi_set_color3() | |
111 | keywords: format | |
112 | ||
113 | The script prints a table showing the available attributes (bold, | |
114 | underline, and inverse) with color combinations for the | |
115 | ans_set_color3() function in the ansi.stp tapset. | |
116 | ||
117 | ||
0e4901b0 WC |
118 | = FREE = |
119 | ||
120 | network/dropwatch.stp - Watch Where Socket Buffers are Freed in the Kernel | |
121 | keywords: network tracepoint buffer free | |
122 | ||
123 | Every five seconds the dropwatch.stp script lists the number of | |
124 | socket buffers freed at locations in the kernel. | |
125 | ||
126 | ||
210ff7bf FCE |
127 | = FUNCTIONS = |
128 | ||
129 | profiling/functioncallcount.stp - Count Times Functions Called | |
09a98ae0 | 130 | keywords: profiling functions |
210ff7bf FCE |
131 | |
132 | The functioncallcount.stp script takes one argument, a list of | |
133 | functions to probe. The script will run and count the number of times | |
134 | that each of the functions on the list is called. On exit the script | |
135 | will print a sorted list from most frequently to least frequently | |
136 | called function. | |
137 | ||
138 | ||
139 | = FUTEX = | |
140 | ||
141 | process/futexes.stp - System-Wide Futex Contention | |
09a98ae0 | 142 | keywords: syscall locking futex |
210ff7bf FCE |
143 | |
144 | The script watches the futex syscall on the system. On exit the | |
145 | futexes address, the number of contentions, and the average time for | |
146 | each contention on the futex are printed from lowest pid number to | |
147 | highest. | |
148 | ||
149 | ||
150 | = GRAPH = | |
151 | ||
152 | general/graphs.stp - Graphing Disk and CPU Utilization | |
09a98ae0 | 153 | keywords: disk cpu use graph |
210ff7bf FCE |
154 | |
155 | The script tracks the disk and CPU utilization. The resulting output | |
156 | of the script can be piped into gnuplot to generate a graph of disk | |
157 | and CPU USE. | |
158 | ||
159 | ||
1cc8a4c7 WC |
160 | = INTERRUPT = |
161 | ||
162 | interrupt/scf.stp - Tally Backtraces for Inter-Processor Interrupt (IPI) | |
163 | keywords: interrupt backtrace | |
164 | ||
165 | The Linux kernel function smp_call_function causes expensive | |
166 | inter-processor interrupts (IPIs). The scf.stp script tallies the | |
167 | processes and backtraces causing the interprocessor interrupts to | |
168 | identify the cause of the expensive IPI. On exit the script prints | |
169 | the tallies in descending frequency. | |
170 | ||
171 | ||
210ff7bf FCE |
172 | = IO = |
173 | ||
174 | io/io_submit.stp - Tally Reschedule Reason During AIO io_submit Call | |
09a98ae0 | 175 | keywords: io backtrace |
210ff7bf FCE |
176 | |
177 | When a reschedule occurs during an AIO io_submit call, accumulate the | |
178 | traceback in a histogram. When the script exits prints out a sorted | |
179 | list from most common to least common backtrace. | |
180 | ||
181 | ||
67f8611b WC |
182 | io/ioblktime.stp - Average Time Block IO Requests Spend in Queue |
183 | keywords: io | |
184 | ||
185 | The ioblktime.stp script tracks the amount of time that each block IO | |
cc20d853 ET |
186 | requests spend waiting for completion. The script computes the |
187 | average time waiting time for block IO per device and prints list | |
188 | every 10 seconds. In some cases there can be too many oustanding | |
189 | block IO operations and the script may exceed the default number of | |
67f8611b WC |
190 | MAXMAPENTRIES allowed. In this case the allowed number can be |
191 | increased with "-DMAXMAPENTRIES=10000" option on the stap command | |
192 | line. | |
193 | ||
194 | ||
54ff5e0c FCE |
195 | io/iostat-scsi.stp - iostat for SCSI Devices |
196 | keywords: io profiling scsi | |
197 | ||
198 | The iostat-scsi.stp script provides a breakdown of the number of blks | |
199 | read and written on the various machines's SCSI devices. The script | |
200 | takes one argument which is the number of seconds between reports. | |
201 | ||
202 | ||
3e4444ed WC |
203 | io/iostats.stp - List Executables Reading and Writing the Most Data |
204 | keywords: io profiling | |
205 | ||
206 | The iostat.stp script measures the amount of data successfully read | |
207 | and written by all the executables on the system. The output is | |
208 | sorted from most greatest sum of bytes read and written by an | |
209 | executable to the least. The output contains the count of operations | |
210 | (opens, reads, and writes), the totals and averages for the number of | |
211 | bytes read and written. | |
212 | ||
213 | ||
210ff7bf | 214 | io/iotime.stp - Trace Time Spent in Read and Write for Files |
09a98ae0 | 215 | keywords: syscall read write time io |
210ff7bf FCE |
216 | |
217 | The script watches each open, close, read, and write syscalls on the | |
218 | system. For each file the scripts observes opened it accumulates the | |
219 | amount of wall clock time spend in read and write operations and the | |
220 | number of bytes read and written. When a file is closed the script | |
221 | prints out a pair of lines for the file. Both lines begin with a | |
222 | timestamp in microseconds, the PID number, and the executable name in | |
223 | parenthesese. The first line with the "access" keyword lists the file | |
224 | name, the attempted number of bytes for the read and write | |
225 | operations. The second line with the "iotime" keyword list the file | |
226 | name and the number of microseconds accumulated in the read and write | |
227 | syscalls. | |
228 | ||
229 | ||
230 | io/iotop.stp - Periodically Print I/O Activity by Process Name | |
09a98ae0 | 231 | keywords: io |
210ff7bf FCE |
232 | |
233 | Every five seconds print out the top ten executables generating I/O | |
234 | traffic during that interval sorted in descending order. | |
235 | ||
236 | ||
237 | io/traceio.stp - Track Cumulative I/O Activity by Process Name | |
09a98ae0 | 238 | keywords: io |
210ff7bf FCE |
239 | |
240 | Every second print out the top ten executables sorted in descending | |
241 | order based on cumulative I/O traffic observed. | |
242 | ||
243 | ||
244 | io/traceio2.stp - Watch I/O Activity on a Particular Device | |
09a98ae0 | 245 | keywords: io |
210ff7bf FCE |
246 | |
247 | Print out the executable name and process number as reads and writes | |
248 | to the specified device occur. | |
249 | ||
250 | ||
b7f6cfc5 FCE |
251 | io/ttyspy.stp - Monitor tty typing. |
252 | keywords: io tty per-process monitor | |
253 | ||
254 | The ttyspy.stp script uses tty_audit hooks to monitor recent typing | |
255 | activity on the system, printing a scrolling record of recent | |
256 | keystrokes, on a per-tty basis. | |
257 | ||
258 | ||
210ff7bf | 259 | process/sleepingBeauties.stp - Generating Backtraces of Threads Waiting for IO Operations |
09a98ae0 | 260 | keywords: io scheduler backtrace |
210ff7bf | 261 | |
cae71dd3 FCE |
262 | The script monitors the time that threads spend waiting for IO |
263 | operations (in "D" state) in the wait_for_completion function. If a | |
264 | thread spends over 10ms, its name and backtrace is printed, and later | |
265 | so is the total delay. | |
210ff7bf FCE |
266 | |
267 | ||
268 | = LOCKING = | |
269 | ||
cf5023fb WC |
270 | locks/bkl.stp - Tracing Contention on Big Kernel Lock (BKL) |
271 | keywords: locking | |
272 | ||
273 | The bkl.stp script can help determine whether the Big Kernel Lock | |
274 | (BKL) is causing serialization on a multiprocessor system due to | |
275 | excessive contention of the BKL. The bkl.stp script takes one | |
276 | argument which is the number of processes waiting for the Big Kernel | |
277 | Lock (BKL). When the number of processes waiting for the BKL is | |
278 | reached or exceeded, the script will print a time stamp, the number | |
279 | of processes waiting for the BKL, the holder of the BKL, and the | |
280 | amount of time the BKL was held. | |
281 | ||
282 | ||
283 | locks/bkl_stats.stp - Per Process Statistics on Big Kernel Lock (BKL) Use | |
284 | keywords: locking | |
285 | ||
286 | The bkl_stats.stp script can indicate which processes have excessive | |
287 | waits for the Big Kernel Lock (BKL) and which processes are taking | |
288 | the BKL for long periods of time. The bkl_stats.stp script prints | |
289 | lists of all the processes that require the BKL. Every five seconds | |
290 | two tables are printed out. The first table lists the processes that | |
291 | waited for the BKL followed by the number of times that the process | |
292 | waited, the minimum time of the wait, the average and the maximum | |
293 | time waited. The second table lists has similar information for the | |
294 | time spent holding the lock for each of the processes. | |
295 | ||
296 | ||
210ff7bf | 297 | process/futexes.stp - System-Wide Futex Contention |
09a98ae0 | 298 | keywords: syscall locking futex |
210ff7bf FCE |
299 | |
300 | The script watches the futex syscall on the system. On exit the | |
301 | futexes address, the number of contentions, and the average time for | |
302 | each contention on the futex are printed from lowest pid number to | |
303 | highest. | |
304 | ||
305 | ||
22f971e8 WC |
306 | = MEMORY = |
307 | ||
308 | memory/kmalloc-top - Show Paths to Kernel Malloc (kmalloc) Invocations | |
309 | keywords: memory | |
310 | ||
311 | The kmalloc-top perl program runs a small systemtap script to collect | |
312 | stack traces for each call to the kmalloc function and counts the | |
313 | time that each stack trace is observed. When kmalloc-top exits it | |
314 | prints out sorted list. The output can be be filtered to print only | |
315 | only the first stack traces (-t) stack traces with more a minimum | |
316 | counts (-m), or exclude certain stack traces (-e). | |
317 | ||
318 | ||
0dc23d1d WC |
319 | memory/mmanonpage.stp - Track Virtual Memory System Actions on Anonymous Pages |
320 | keywords: memory | |
321 | ||
322 | The mmanonpage.stp script uses the virtual memory tracepoints | |
323 | available in some kernels to track the number of faults, user space | |
324 | frees, page ins, copy on writes and unmaps for anonymous pages. When | |
325 | the script is terminated the counts are printed for each process that | |
326 | allocated pages while the script was running. This script displays | |
327 | the anonymous page statistics for each process that ran while the | |
328 | script is active. Its useful in debugging leaks in the anonymous | |
329 | regions of a process. | |
330 | ||
331 | ||
332 | memory/mmfilepage.stp - Track Virtual Memory System Actions on File Backed Pages | |
333 | keywords: memory | |
334 | ||
335 | The mmfilepage.stp script uses the virtual memory tracepoints | |
336 | available in some kernels to track the number of faults, copy on | |
337 | writes mapping, and unmapping operations for file backed pages. When | |
338 | the script is terminated the counts are printed for each process that | |
339 | allocated pages while the script was running. The mmfilepage.stp | |
340 | script is useful in debugging leaks in the mapped file regions of a | |
341 | process. | |
342 | ||
343 | ||
344 | memory/mmreclaim.stp - Track Virtual Memory System Page Reclamation | |
345 | keywords: memory | |
346 | ||
347 | The mmreclaim.stp script uses the virtual memory tracepoints | |
348 | available in some kernels to track page reclaim activity that occured | |
349 | while the script was running. Its useful is debugging performance | |
350 | problems that occur due to page reclamation. | |
351 | ||
352 | ||
353 | memory/mmwriteback.stp - Track Virtual Memory System Writing to Disk | |
354 | keywords: memory | |
355 | ||
356 | The mmwriteback.stp script uses the virtual memory tracepoints | |
357 | available in some kernels to report all of the file writebacks that | |
358 | occur form kupdate, pdflush and kjournald while the script is | |
359 | running. Its useful in determining where writes are coming from on a | |
360 | supposedly idle system that is experiencing upexpected IO. | |
361 | ||
362 | ||
c728b7da WC |
363 | memory/numa_faults.stp - Summarize Process Misses across NUMA Nodes |
364 | keywords: memory numa | |
365 | ||
366 | The numa_faults.stp script tracks the read and write pages faults for | |
367 | each process. When the script exits it prints out the total read and | |
368 | write pages faults for each process. The script also providea a break | |
369 | down of page faults per node for each process. This script is useful | |
370 | for determining whether the program has good locality (page faults | |
371 | limited to a single node) on a NUMA computer. | |
372 | ||
373 | ||
413996e0 WC |
374 | memory/pfaults.stp - Generate Log of Major and Minor Page Faults |
375 | keywords: memory | |
376 | ||
377 | The pfaults.stp script generates a simple log for each major and | |
378 | minor page fault that occurs on the system. Each line contains a | |
379 | timestamp (in microseconds) when the page fault servicing was | |
380 | completed, the pid of the process, the address of the page fault, the | |
381 | type of access (read or write), the type of fault (major or minor), | |
382 | and the elapsed time for page fault. This log can be examined to | |
383 | determine where the page faults are occuring. | |
384 | ||
385 | ||
905728a0 WC |
386 | network/sk_stream_wait_memory.stp - Track Start and Stop of Processes Due to Network Buffer Space |
387 | keywords: network tcp buffer memory | |
388 | ||
389 | The sk_stream-wait_memory.stp prints a time stamp, executable, and | |
390 | pid each time a process blocks due to the send buffer being full. A | |
391 | similar entry is printed each time a process continues because there | |
392 | is room in the buffer. | |
393 | ||
394 | ||
b7f6cfc5 FCE |
395 | = MONITOR = |
396 | ||
397 | io/ttyspy.stp - Monitor tty typing. | |
398 | keywords: io tty per-process monitor | |
399 | ||
400 | The ttyspy.stp script uses tty_audit hooks to monitor recent typing | |
401 | activity on the system, printing a scrolling record of recent | |
402 | keystrokes, on a per-tty basis. | |
403 | ||
404 | ||
210ff7bf FCE |
405 | = NETWORK = |
406 | ||
8b88e771 FCE |
407 | network/autofs4.stp - Watch autofs4 operations |
408 | keywords: network autofs nfs | |
409 | ||
410 | Trace key autofs4 operations such as mounting or unmounting remote | |
411 | filesystems. | |
412 | ||
413 | ||
0e4901b0 WC |
414 | network/dropwatch.stp - Watch Where Socket Buffers are Freed in the Kernel |
415 | keywords: network tracepoint buffer free | |
416 | ||
417 | Every five seconds the dropwatch.stp script lists the number of | |
418 | socket buffers freed at locations in the kernel. | |
419 | ||
420 | ||
210ff7bf | 421 | network/nettop.stp - Periodic Listing of Processes Using Network Interfaces |
09a98ae0 | 422 | keywords: network traffic per-process |
210ff7bf FCE |
423 | |
424 | Every five seconds the nettop.stp script prints out a list of | |
425 | processed (PID and command) with the number of packets sent/received | |
426 | and the amount of data sent/received by the process during that | |
427 | interval. | |
428 | ||
429 | ||
905728a0 WC |
430 | network/sk_stream_wait_memory.stp - Track Start and Stop of Processes Due to Network Buffer Space |
431 | keywords: network tcp buffer memory | |
432 | ||
433 | The sk_stream-wait_memory.stp prints a time stamp, executable, and | |
434 | pid each time a process blocks due to the send buffer being full. A | |
435 | similar entry is printed each time a process continues because there | |
436 | is room in the buffer. | |
437 | ||
438 | ||
210ff7bf | 439 | network/socket-trace.stp - Trace Functions called in Network Socket Code |
09a98ae0 | 440 | keywords: network socket |
210ff7bf FCE |
441 | |
442 | The script instrument each of the functions inn the Linux kernel's | |
443 | net/socket.c file. The script prints out trace. The first element of | |
444 | a line is time delta in microseconds from the previous entry. This | |
445 | is followed by the command name and the PID. The "->" and "<-" | |
446 | indicates function entry and function exit, respectively. The last | |
447 | element of the line is the function name. | |
448 | ||
449 | ||
492d227f WC |
450 | network/tcp_connections.stp - Track Creation of Incoming TCP Connections |
451 | keywords: network tcp socket | |
452 | ||
453 | The tcp_connections.stp script prints information for each new | |
454 | incoming TCP connection accepted by the computer. The information | |
455 | includes the UID, the command accepting the connection, the PID of | |
456 | the command, the port the connection is on, and the IP address of the | |
457 | originator of the request. | |
458 | ||
459 | ||
7a51212c WC |
460 | network/tcpdumplike.stp - Dump of Received TCP Packets |
461 | keywords: network traffic | |
462 | ||
463 | The tcpdumplike.stp prints out a line for each TCP packet received. | |
464 | Each line includes the source and destination IP addresses, the | |
465 | source and destination ports, and flags. | |
466 | ||
467 | ||
4bb6522c WC |
468 | network/tcpipstat.stp - Display network statistics for individual TCP sockets. |
469 | keywords: network statistics | |
470 | ||
471 | tcpipstat collects and display network statistics related to | |
472 | individual TCP sockets or groups of sockets. The statistics that are | |
473 | collected are simmer to that of the command netstat -s, only sorted | |
474 | and grouped by individual sockets. | |
475 | ||
476 | ||
8b88e771 FCE |
477 | = NFS = |
478 | ||
479 | network/autofs4.stp - Watch autofs4 operations | |
480 | keywords: network autofs nfs | |
481 | ||
482 | Trace key autofs4 operations such as mounting or unmounting remote | |
483 | filesystems. | |
484 | ||
485 | ||
c728b7da WC |
486 | = NUMA = |
487 | ||
488 | memory/numa_faults.stp - Summarize Process Misses across NUMA Nodes | |
489 | keywords: memory numa | |
490 | ||
491 | The numa_faults.stp script tracks the read and write pages faults for | |
492 | each process. When the script exits it prints out the total read and | |
493 | write pages faults for each process. The script also providea a break | |
494 | down of page faults per node for each process. This script is useful | |
495 | for determining whether the program has good locality (page faults | |
496 | limited to a single node) on a NUMA computer. | |
497 | ||
498 | ||
210ff7bf FCE |
499 | = PER-PROCESS = |
500 | ||
b7f6cfc5 FCE |
501 | io/ttyspy.stp - Monitor tty typing. |
502 | keywords: io tty per-process monitor | |
503 | ||
504 | The ttyspy.stp script uses tty_audit hooks to monitor recent typing | |
505 | activity on the system, printing a scrolling record of recent | |
506 | keystrokes, on a per-tty basis. | |
507 | ||
508 | ||
210ff7bf | 509 | network/nettop.stp - Periodic Listing of Processes Using Network Interfaces |
09a98ae0 | 510 | keywords: network traffic per-process |
210ff7bf FCE |
511 | |
512 | Every five seconds the nettop.stp script prints out a list of | |
513 | processed (PID and command) with the number of packets sent/received | |
514 | and the amount of data sent/received by the process during that | |
515 | interval. | |
516 | ||
517 | ||
cc20d853 ET |
518 | = PROCESS = |
519 | ||
520 | process/errsnoop.stp - tabulate system call errors | |
521 | keywords: process syscall | |
522 | ||
523 | The script prints a periodic tabular report about failing system | |
524 | calls, by process and by syscall failure. The first optional | |
525 | argument specifies the reporting interval (in seconds, default 5); | |
526 | the second optional argument gives a screen height (number of lines | |
527 | in the report, default 20). | |
528 | ||
529 | ||
e6b653c8 WC |
530 | process/forktracker.stp - Trace Creation of Processes |
531 | keywords: process scheduler | |
532 | ||
533 | The forktracker.stp script prints out a time-stamped entry showing | |
534 | each fork and exec operation on the machine. This can be useful for | |
535 | determine what process is creating a flurry of short-lived processes. | |
536 | ||
537 | ||
34029cd3 WC |
538 | process/schedtimes.stp - Track Time Processes Spend in Various States using Tracepoints |
539 | keywords: process scheduler time tracepoint | |
540 | ||
541 | The schedtimes.stp script instruments the scheduler to track the | |
542 | amount of time that each process spends running, sleeping, queued, | |
543 | and waiting for io. On exit the script prints out the accumulated | |
544 | time for each state of processes observed. Optionally, this script | |
545 | can be used with the '-c' or '-x' options to focus on a specific PID. | |
546 | ||
547 | ||
210ff7bf FCE |
548 | = PROFILING = |
549 | ||
54ff5e0c FCE |
550 | io/iostat-scsi.stp - iostat for SCSI Devices |
551 | keywords: io profiling scsi | |
552 | ||
553 | The iostat-scsi.stp script provides a breakdown of the number of blks | |
554 | read and written on the various machines's SCSI devices. The script | |
555 | takes one argument which is the number of seconds between reports. | |
556 | ||
557 | ||
3e4444ed WC |
558 | io/iostats.stp - List Executables Reading and Writing the Most Data |
559 | keywords: io profiling | |
560 | ||
561 | The iostat.stp script measures the amount of data successfully read | |
562 | and written by all the executables on the system. The output is | |
563 | sorted from most greatest sum of bytes read and written by an | |
564 | executable to the least. The output contains the count of operations | |
565 | (opens, reads, and writes), the totals and averages for the number of | |
566 | bytes read and written. | |
567 | ||
568 | ||
210ff7bf | 569 | process/pf2.stp - Profile kernel functions |
09a98ae0 | 570 | keywords: profiling |
210ff7bf FCE |
571 | |
572 | The pf2.stp script sets up time-based sampling. Every five seconds it | |
573 | prints out a sorted list with the top ten kernel functions with | |
574 | samples. | |
575 | ||
576 | ||
577 | profiling/functioncallcount.stp - Count Times Functions Called | |
09a98ae0 | 578 | keywords: profiling functions |
210ff7bf FCE |
579 | |
580 | The functioncallcount.stp script takes one argument, a list of | |
581 | functions to probe. The script will run and count the number of times | |
582 | that each of the functions on the list is called. On exit the script | |
583 | will print a sorted list from most frequently to least frequently | |
584 | called function. | |
585 | ||
586 | ||
587 | profiling/thread-times.stp - Profile kernel functions | |
09a98ae0 | 588 | keywords: profiling |
210ff7bf FCE |
589 | |
590 | The thread-times.stp script sets up time-based sampling. Every five | |
591 | seconds it prints out a sorted list with the top twenty processes | |
592 | with samples broken down into percentage total time spent in | |
593 | user-space and kernel-space. | |
594 | ||
595 | ||
ff90b297 WC |
596 | profiling/timeout.stp - Show Processes Doing Polling Operations |
597 | keywords: profiling | |
598 | ||
599 | The timeout.stp script is based on a blog entry | |
600 | (http://udrepper.livejournal.com/19041.html) mentioning a need for a | |
601 | tool to help developers find applications that are polling. The | |
602 | timeout.stp script monitors systemcall used for polling and records | |
603 | the systemcalls that timed out rather than returned because some | |
604 | action occurred. The script updates the screen once a second with the | |
605 | top twenty processes. | |
606 | ||
607 | ||
1bf72dfe WC |
608 | profiling/topsys.stp - Show Processes Doing Polling Operations |
609 | keywords: profiling | |
610 | ||
611 | The topsys.stp script lists out the top twenty systemcalls for the | |
612 | previous 5 seconds. The output is sorted from most frequent to least | |
613 | frequent. | |
614 | ||
615 | ||
210ff7bf FCE |
616 | = READ = |
617 | ||
618 | io/iotime.stp - Trace Time Spent in Read and Write for Files | |
09a98ae0 | 619 | keywords: syscall read write time io |
210ff7bf FCE |
620 | |
621 | The script watches each open, close, read, and write syscalls on the | |
622 | system. For each file the scripts observes opened it accumulates the | |
623 | amount of wall clock time spend in read and write operations and the | |
624 | number of bytes read and written. When a file is closed the script | |
625 | prints out a pair of lines for the file. Both lines begin with a | |
626 | timestamp in microseconds, the PID number, and the executable name in | |
627 | parenthesese. The first line with the "access" keyword lists the file | |
628 | name, the attempted number of bytes for the read and write | |
629 | operations. The second line with the "iotime" keyword list the file | |
630 | name and the number of microseconds accumulated in the read and write | |
631 | syscalls. | |
632 | ||
633 | ||
634 | = SCHEDULER = | |
635 | ||
deb63545 WC |
636 | process/chng_cpu.stp - Monitor Changes in Processor Executing a Task |
637 | keywords: scheduler | |
638 | ||
639 | The chng_cpu.stp script takes an argument which is the executable | |
640 | name of the task it should monitor. Each time a task with that | |
641 | executable name is found running on a different processor, the script | |
642 | prints out the thread id (tid), the executable name, the processor | |
643 | now running the task, the thread state, and a backtrace showing the | |
644 | kernel functions that triggered the running of the task on the | |
645 | processor. | |
646 | ||
647 | ||
e6b653c8 WC |
648 | process/forktracker.stp - Trace Creation of Processes |
649 | keywords: process scheduler | |
650 | ||
651 | The forktracker.stp script prints out a time-stamped entry showing | |
652 | each fork and exec operation on the machine. This can be useful for | |
653 | determine what process is creating a flurry of short-lived processes. | |
654 | ||
655 | ||
deb63545 WC |
656 | process/migrate.stp - Track the Migration of Specific Executables |
657 | keywords: scheduler | |
658 | ||
659 | The migrate.stp script takes an argument which is the executable name | |
660 | of the task it should monitor. Each time a task with that executable | |
661 | name migrates between processors an entry is printed with the process | |
662 | id (pid), the executable name, the processor off loading the task, | |
663 | and the process taking the task. Note that the task may or may not be | |
664 | executing at the time of the migration. | |
665 | ||
666 | ||
34029cd3 WC |
667 | process/schedtimes.stp - Track Time Processes Spend in Various States using Tracepoints |
668 | keywords: process scheduler time tracepoint | |
669 | ||
670 | The schedtimes.stp script instruments the scheduler to track the | |
671 | amount of time that each process spends running, sleeping, queued, | |
672 | and waiting for io. On exit the script prints out the accumulated | |
673 | time for each state of processes observed. Optionally, this script | |
674 | can be used with the '-c' or '-x' options to focus on a specific PID. | |
675 | ||
676 | ||
210ff7bf | 677 | process/sleepingBeauties.stp - Generating Backtraces of Threads Waiting for IO Operations |
09a98ae0 | 678 | keywords: io scheduler backtrace |
cae71dd3 FCE |
679 | |
680 | The script monitors the time that threads spend waiting for IO | |
681 | operations (in "D" state) in the wait_for_completion function. If a | |
682 | thread spends over 10ms, its name and backtrace is printed, and later | |
683 | so is the total delay. | |
210ff7bf FCE |
684 | |
685 | ||
54ff5e0c FCE |
686 | = SCSI = |
687 | ||
688 | io/iostat-scsi.stp - iostat for SCSI Devices | |
689 | keywords: io profiling scsi | |
690 | ||
691 | The iostat-scsi.stp script provides a breakdown of the number of blks | |
692 | read and written on the various machines's SCSI devices. The script | |
693 | takes one argument which is the number of seconds between reports. | |
694 | ||
695 | ||
210ff7bf FCE |
696 | = SIGNALS = |
697 | ||
698 | process/sig_by_pid.stp - Signal Counts by Process ID | |
09a98ae0 | 699 | keywords: signals |
210ff7bf FCE |
700 | |
701 | Print signal counts by process ID in descending order. | |
702 | ||
703 | ||
704 | process/sig_by_proc.stp - Signal Counts by Process Name | |
09a98ae0 | 705 | keywords: signals |
210ff7bf FCE |
706 | |
707 | Print signal counts by process name in descending order. | |
708 | ||
709 | ||
710 | process/sigkill.stp - Track SIGKILL Signals | |
09a98ae0 | 711 | keywords: signals |
210ff7bf FCE |
712 | |
713 | The script traces any SIGKILL signals. When that SIGKILL signal is | |
714 | sent to a process, the script prints out the signal name, the | |
715 | desination executable and process ID, the executable name user ID | |
716 | that sent the signal. | |
717 | ||
718 | ||
3a748561 | 719 | process/sigmon.stp - Track a particular signal to a specific process |
09a98ae0 | 720 | keywords: signals |
210ff7bf FCE |
721 | |
722 | The script watches for a particular signal sent to a specific | |
723 | process. When that signal is sent to the specified process, the | |
724 | script prints out the PID and executable of the process sending the | |
725 | signal, the PID and executable name of the process receiving the | |
726 | signal, and the signal number and name. | |
727 | ||
728 | ||
729 | = SIMPLE = | |
730 | ||
731 | general/helloworld.stp - SystemTap "Hello World" Program | |
09a98ae0 | 732 | keywords: simple |
210ff7bf FCE |
733 | |
734 | A basic "Hello World" program implemented in SystemTap script. It | |
735 | prints out "hello world" message and then immediately exits. | |
736 | ||
737 | ||
738 | = SLEEP = | |
739 | ||
740 | process/sleeptime.stp - Trace Time Spent in nanosleep Syscalls | |
09a98ae0 | 741 | keywords: syscall sleep |
210ff7bf FCE |
742 | |
743 | The script watches each nanosleep syscall on the system. At the end | |
744 | of each nanosleep syscall the script prints out a line with a | |
745 | timestamp in microseconds, the pid, the executable name in | |
746 | paretheses, the "nanosleep:" key, and the duration of the sleep in | |
747 | microseconds. | |
748 | ||
749 | ||
750 | = SOCKET = | |
751 | ||
752 | network/socket-trace.stp - Trace Functions called in Network Socket Code | |
09a98ae0 | 753 | keywords: network socket |
210ff7bf FCE |
754 | |
755 | The script instrument each of the functions inn the Linux kernel's | |
756 | net/socket.c file. The script prints out trace. The first element of | |
757 | a line is time delta in microseconds from the previous entry. This | |
758 | is followed by the command name and the PID. The "->" and "<-" | |
759 | indicates function entry and function exit, respectively. The last | |
760 | element of the line is the function name. | |
761 | ||
762 | ||
492d227f WC |
763 | network/tcp_connections.stp - Track Creation of Incoming TCP Connections |
764 | keywords: network tcp socket | |
765 | ||
766 | The tcp_connections.stp script prints information for each new | |
767 | incoming TCP connection accepted by the computer. The information | |
768 | includes the UID, the command accepting the connection, the PID of | |
769 | the command, the port the connection is on, and the IP address of the | |
770 | originator of the request. | |
771 | ||
772 | ||
4bb6522c WC |
773 | = STATISTICS = |
774 | ||
775 | network/tcpipstat.stp - Display network statistics for individual TCP sockets. | |
776 | keywords: network statistics | |
777 | ||
778 | tcpipstat collects and display network statistics related to | |
779 | individual TCP sockets or groups of sockets. The statistics that are | |
780 | collected are simmer to that of the command netstat -s, only sorted | |
781 | and grouped by individual sockets. | |
782 | ||
783 | ||
210ff7bf FCE |
784 | = SYSCALL = |
785 | ||
786 | io/iotime.stp - Trace Time Spent in Read and Write for Files | |
09a98ae0 | 787 | keywords: syscall read write time io |
210ff7bf FCE |
788 | |
789 | The script watches each open, close, read, and write syscalls on the | |
790 | system. For each file the scripts observes opened it accumulates the | |
791 | amount of wall clock time spend in read and write operations and the | |
792 | number of bytes read and written. When a file is closed the script | |
793 | prints out a pair of lines for the file. Both lines begin with a | |
794 | timestamp in microseconds, the PID number, and the executable name in | |
795 | parenthesese. The first line with the "access" keyword lists the file | |
796 | name, the attempted number of bytes for the read and write | |
797 | operations. The second line with the "iotime" keyword list the file | |
798 | name and the number of microseconds accumulated in the read and write | |
799 | syscalls. | |
800 | ||
801 | ||
cc20d853 ET |
802 | process/errsnoop.stp - tabulate system call errors |
803 | keywords: process syscall | |
804 | ||
805 | The script prints a periodic tabular report about failing system | |
806 | calls, by process and by syscall failure. The first optional | |
807 | argument specifies the reporting interval (in seconds, default 5); | |
808 | the second optional argument gives a screen height (number of lines | |
809 | in the report, default 20). | |
810 | ||
811 | ||
210ff7bf | 812 | process/futexes.stp - System-Wide Futex Contention |
09a98ae0 | 813 | keywords: syscall locking futex |
210ff7bf FCE |
814 | |
815 | The script watches the futex syscall on the system. On exit the | |
816 | futexes address, the number of contentions, and the average time for | |
817 | each contention on the futex are printed from lowest pid number to | |
818 | highest. | |
819 | ||
820 | ||
821 | process/sleeptime.stp - Trace Time Spent in nanosleep Syscalls | |
09a98ae0 | 822 | keywords: syscall sleep |
210ff7bf FCE |
823 | |
824 | The script watches each nanosleep syscall on the system. At the end | |
825 | of each nanosleep syscall the script prints out a line with a | |
826 | timestamp in microseconds, the pid, the executable name in | |
827 | paretheses, the "nanosleep:" key, and the duration of the sleep in | |
828 | microseconds. | |
829 | ||
830 | ||
831 | process/syscalls_by_pid.stp - System-Wide Count of Syscalls by PID | |
09a98ae0 | 832 | keywords: syscall |
210ff7bf FCE |
833 | |
834 | The script watches all syscall on the system. On exit the script | |
835 | prints a list showing the number of systemcalls executed by each PID | |
836 | ordered from greatest to least number of syscalls. | |
837 | ||
838 | ||
839 | process/syscalls_by_proc.stp - System-Wide Count of Syscalls by Executable | |
09a98ae0 | 840 | keywords: syscall |
210ff7bf FCE |
841 | |
842 | The script watches all syscall on the system. On exit the script | |
843 | prints a list showing the number of systemcalls executed by each | |
844 | executable ordered from greates to least number of syscalls. | |
845 | ||
846 | ||
847 | process/wait4time.stp - Trace Time Spent in wait4 Syscalls | |
09a98ae0 | 848 | keywords: syscall wait4 |
210ff7bf FCE |
849 | |
850 | The script watches each wait4 syscall on the system. At the end of | |
851 | each wait4 syscall the script prints out a line with a timestamp in | |
852 | microseconds, the pid, the executable name in paretheses, the | |
853 | "wait4:" key, the duration of the wait and the PID that the wait4 was | |
854 | waiting for. If the waited for PID is not specified , it is "-1". | |
855 | ||
856 | ||
492d227f WC |
857 | = TCP = |
858 | ||
905728a0 WC |
859 | network/sk_stream_wait_memory.stp - Track Start and Stop of Processes Due to Network Buffer Space |
860 | keywords: network tcp buffer memory | |
861 | ||
862 | The sk_stream-wait_memory.stp prints a time stamp, executable, and | |
863 | pid each time a process blocks due to the send buffer being full. A | |
864 | similar entry is printed each time a process continues because there | |
865 | is room in the buffer. | |
866 | ||
867 | ||
492d227f WC |
868 | network/tcp_connections.stp - Track Creation of Incoming TCP Connections |
869 | keywords: network tcp socket | |
870 | ||
871 | The tcp_connections.stp script prints information for each new | |
872 | incoming TCP connection accepted by the computer. The information | |
873 | includes the UID, the command accepting the connection, the PID of | |
874 | the command, the port the connection is on, and the IP address of the | |
875 | originator of the request. | |
876 | ||
877 | ||
210ff7bf FCE |
878 | = TIME = |
879 | ||
880 | io/iotime.stp - Trace Time Spent in Read and Write for Files | |
09a98ae0 | 881 | keywords: syscall read write time io |
210ff7bf FCE |
882 | |
883 | The script watches each open, close, read, and write syscalls on the | |
884 | system. For each file the scripts observes opened it accumulates the | |
885 | amount of wall clock time spend in read and write operations and the | |
886 | number of bytes read and written. When a file is closed the script | |
887 | prints out a pair of lines for the file. Both lines begin with a | |
888 | timestamp in microseconds, the PID number, and the executable name in | |
889 | parenthesese. The first line with the "access" keyword lists the file | |
890 | name, the attempted number of bytes for the read and write | |
891 | operations. The second line with the "iotime" keyword list the file | |
892 | name and the number of microseconds accumulated in the read and write | |
893 | syscalls. | |
894 | ||
895 | ||
34029cd3 WC |
896 | process/schedtimes.stp - Track Time Processes Spend in Various States using Tracepoints |
897 | keywords: process scheduler time tracepoint | |
898 | ||
899 | The schedtimes.stp script instruments the scheduler to track the | |
900 | amount of time that each process spends running, sleeping, queued, | |
901 | and waiting for io. On exit the script prints out the accumulated | |
902 | time for each state of processes observed. Optionally, this script | |
903 | can be used with the '-c' or '-x' options to focus on a specific PID. | |
904 | ||
905 | ||
210ff7bf FCE |
906 | = TRACE = |
907 | ||
aa15b9f0 | 908 | general/para-callgraph.stp - Callgraph tracing with arguments |
09a98ae0 | 909 | keywords: trace callgraph |
210ff7bf | 910 | |
aa15b9f0 FCE |
911 | Print a timed per-thread callgraph, complete with function parameters |
912 | and return values. The first parameter names the function probe | |
913 | points to trace. The optional second parameter names the probe | |
914 | points for trigger functions, which acts to enable tracing for only | |
915 | those functions that occur while the current thread is nested within | |
916 | the trigger. | |
210ff7bf FCE |
917 | |
918 | ||
0e4901b0 WC |
919 | = TRACEPOINT = |
920 | ||
921 | network/dropwatch.stp - Watch Where Socket Buffers are Freed in the Kernel | |
922 | keywords: network tracepoint buffer free | |
923 | ||
924 | Every five seconds the dropwatch.stp script lists the number of | |
925 | socket buffers freed at locations in the kernel. | |
926 | ||
927 | ||
34029cd3 WC |
928 | process/schedtimes.stp - Track Time Processes Spend in Various States using Tracepoints |
929 | keywords: process scheduler time tracepoint | |
930 | ||
931 | The schedtimes.stp script instruments the scheduler to track the | |
932 | amount of time that each process spends running, sleeping, queued, | |
933 | and waiting for io. On exit the script prints out the accumulated | |
934 | time for each state of processes observed. Optionally, this script | |
935 | can be used with the '-c' or '-x' options to focus on a specific PID. | |
936 | ||
937 | ||
210ff7bf FCE |
938 | = TRAFFIC = |
939 | ||
940 | network/nettop.stp - Periodic Listing of Processes Using Network Interfaces | |
09a98ae0 | 941 | keywords: network traffic per-process |
210ff7bf FCE |
942 | |
943 | Every five seconds the nettop.stp script prints out a list of | |
944 | processed (PID and command) with the number of packets sent/received | |
945 | and the amount of data sent/received by the process during that | |
946 | interval. | |
947 | ||
948 | ||
7a51212c WC |
949 | network/tcpdumplike.stp - Dump of Received TCP Packets |
950 | keywords: network traffic | |
951 | ||
952 | The tcpdumplike.stp prints out a line for each TCP packet received. | |
953 | Each line includes the source and destination IP addresses, the | |
954 | source and destination ports, and flags. | |
955 | ||
956 | ||
b7f6cfc5 FCE |
957 | = TTY = |
958 | ||
959 | io/ttyspy.stp - Monitor tty typing. | |
960 | keywords: io tty per-process monitor | |
961 | ||
962 | The ttyspy.stp script uses tty_audit hooks to monitor recent typing | |
963 | activity on the system, printing a scrolling record of recent | |
964 | keystrokes, on a per-tty basis. | |
965 | ||
966 | ||
210ff7bf FCE |
967 | = USE = |
968 | ||
969 | general/graphs.stp - Graphing Disk and CPU Utilization | |
09a98ae0 | 970 | keywords: disk cpu use graph |
210ff7bf FCE |
971 | |
972 | The script tracks the disk and CPU utilization. The resulting output | |
973 | of the script can be piped into gnuplot to generate a graph of disk | |
974 | and CPU USE. | |
975 | ||
976 | ||
977 | = WAIT4 = | |
978 | ||
979 | process/wait4time.stp - Trace Time Spent in wait4 Syscalls | |
09a98ae0 | 980 | keywords: syscall wait4 |
210ff7bf FCE |
981 | |
982 | The script watches each wait4 syscall on the system. At the end of | |
983 | each wait4 syscall the script prints out a line with a timestamp in | |
984 | microseconds, the pid, the executable name in paretheses, the | |
985 | "wait4:" key, the duration of the wait and the PID that the wait4 was | |
986 | waiting for. If the waited for PID is not specified , it is "-1". | |
987 | ||
988 | ||
989 | = WRITE = | |
990 | ||
991 | io/iotime.stp - Trace Time Spent in Read and Write for Files | |
09a98ae0 | 992 | keywords: syscall read write time io |
210ff7bf FCE |
993 | |
994 | The script watches each open, close, read, and write syscalls on the | |
995 | system. For each file the scripts observes opened it accumulates the | |
996 | amount of wall clock time spend in read and write operations and the | |
997 | number of bytes read and written. When a file is closed the script | |
998 | prints out a pair of lines for the file. Both lines begin with a | |
999 | timestamp in microseconds, the PID number, and the executable name in | |
1000 | parenthesese. The first line with the "access" keyword lists the file | |
1001 | name, the attempted number of bytes for the read and write | |
1002 | operations. The second line with the "iotime" keyword list the file | |
1003 | name and the number of microseconds accumulated in the read and write | |
1004 | syscalls. | |
1005 | ||
1006 |