feedback from google

[Chen, Brad]

Earlier we were looking for stronger justification for 
scrutinizing kprobes performance; why "high-frequency" 
probes executing over 5KHz would matter to anybody. I 
was talking with a friend at Google today (Dick Sites)
about Systemtap and he jumped on this immediately as 
an issue. He suggested a couple examples off the top 
of his head of things we'd want to monitor that pushed 
well above the 1KHz event frequency range:
- context switches. Just moving the mouse around on 
the desktop can generate context switches at over 2KHz. 
Peak context switch rates are probably over 100K events 
per second; are we ready?
- network interrupts. A Gb Ethernet receiving full-sized 
packets at full rates would generate about 80K network 
interrupts per second. Smaller packets could be worse.
Actually, if you just do the arithmetic of peak rates 
for things like page faults, disk requests, system calls 
etc. it's easy to find cases that involve interrupt
rates much higher than 5KHz.

Sites was happy to hear about the Systemtap effort and 
sounds like he would eager to give it a try. One tool he 
suggested was to generate a trace of lock, disk, network 
and inter-processor notification events, basically 
everything that could possibly be related to unwanted 
idle time on a performance critical system. Probably not 
realistic for our first system but might be a good test 
case with a faster version of kprobes.

For the record, I have no expectation that kprobes 
performance work can happen in time for the first release.

Brad