djprobes status

[Satoshi Oshima]

Hi Ingo,

Ingo Molnar wrote:
> On Fri, 2006-09-15 at 09:49 -0400, Satoshi Oshima wrote:
>> The problems currently we are trying to solve is...
>>
>> (1)finding the safe place to put probes
>> (2)unregistration of the buffer that is used as out of line execution
>>    under kernel level preemption(CONFIG_PREEMPT=y)
>>
>> We have some idea to solve problem (1), and we are working on it.
>>
>> Currently (2) is very difficult to solve. This problem is equivalent
>> to the problem of finding RCU quiescent state without rcu_read_lock()/
>> rcu_read_unlock(). We need to add a couple of primitives to find such
>> state. But it is not easy. We may be going to push djprobe without 
>> kernel level preemption support like current kprobe-booster.
>>
> 
> could you tell me a bit more about how you have tried to solve (2) and
> why it didnt work? I cannot see any fundamental difficulty in (2), in
> fact i find problem (1) much, much scarier! :-) (please also outline the
> basic problem to me, maybe i'm not seeing the difficulties out of
> ignorance.)

I try to explain two problems above and attach djprobe document
if you are interested in the detail of djprobes. And I appreciate
your help, suggestion or comment!


(1)finding the safe place to put probes

Imagine certain binary line which is constructed by 2 byte instruction, 
2byte instruction and 3byte instruction, like below

         IA
         | 
[-2][-1][0][1][2][3][4][5][6][7]
        [ins1][ins2][  ins3 ]
        [<-     DCR       ->]
           [<- JTPR ->]

ins1: 1st Instruction
ins2: 2nd Instruction
ins3: 3rd Instruction
IA:  Insertion Address
JTPR: Jump Target Prohibition Region
DCR: Detoured Code Region

If we try to put a probe onto IA, we copy all instruction in DCR to
the buffer that is in executable pages and replace 5 byte in DCR
with relative jmp instruction. The problem (1) is the case that
some other part of the kernel might try to jump or call into JTPR. 

Yes, this is scary. But basically, we have to look into all jmp,
call, etc..instructions in the same function. I believe that
there is no possibilities that someone jump into the middle of the
function from other functions. The exception is assembler code.


(2)unregistration of the buffer that is used as out of line execution
   under kernel level preemption(CONFIG_PREEMPT=y)

Djprobes copy all instruction in DCR to the buffer and add another
jmp instruction after that. In the case of unregistration, djprobes
put break point instruction and put original instruction after the
safety check. After that djprobes have to release the buffer. 

The problem is that it is very difficult to find the status that
the buffer is not in use, because someone preempt the execution
in the buffer and preempted thread is in expired and that thread
might be re-balanced to another CPU queue. 

This is very similar to RCU quiescent status. In the case of
RCU, user must make rcu_read_lock()/unlock() when being in
read critical region. rcu_read_lock()/unlock() is implemented
as preempt_disable()/enable().

For djprobe, it is unavailable to take such a strategy, because 
there is no space to put preempt_enable() after jmp instruction
in the buffer or in original code.


Satoshi
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: djprobe-20051031.txt
URL: <http://sourceware.org/pipermail/systemtap/attachments/20060915/b20b46d3/attachment.txt>