Why systemtap loses events and what can be done about it?

[Mike Mason]

I've been toying with the idea of optimizing probe handlers by implementing them  like interrupt handlers, with top and bottom halves.  The top half could cache the minimal data necessary (context, timestamp), allowing the handler to exit as quickly as possible.   The bottom half could do the bulk of the processing (assigning to arrays, calculations, etc.).  Depending on how the bottom half is implemented, it *might* also be able to do things like locking and sleeping.

Of course, since I've never actually written an interrupt handler, I might be blowing smoke.  It's just a thought.

Mike

Frank Ch. Eigler wrote:
> Brad Peters <bpeters@linux.vnet.ibm.com> writes:
> 
>> [...] as interrupts are disabled whenever a probe is run, only one can run
>> at any given time on a per cpu basis. [...]
> 
> We would also block reentrant probes, so disabling interrupts is only
> a means rather than the cause of atomic probe execution.
> 
>> The only solution, AFIK, is to optimize/minimize your probe code.
> 
> Running it with "-t" (timing) will give some measurements about how
> much time the probe handlers are taking, and thus can help guide
> optimization.
> 
>> This does not eliminate the problem, however.  Given enough probe
>> hits, you will still see contention.
> 
> Yes, that is the nature of the beast.
> 
> - FChE