Hitachi djprobe mechanism

[Satoshi Oshima]

Karim Yaghmour wrote:
>>But the case 2), we believe that we can expect currently
>>sleeping process' stack only include EIPs which are limited
>>address such as might_resched() or sched(). So djprobe user
>>must not insert a probe to such point. In my understanding,
>>voluntary or non preemption kernel doesn't try to preempt
>>during interruption context.
> 
> But what about when the call that caused the resched came from higher
> up the call tree and that it is that former call that is getting
> squashed by the insertion of a jump on the instruction preceeding it.

I see.

We should add another limitation to djprobe limitation list.
Current list is ...
------------------------------------------

limitation of djprobe

djprobe user must avoid inserting a probe into the place below:

code includes relative jmp instruction
code includes call instruction
code includes int instruction
functions that preempt current process such as sched() or might_resched()

------------------------------------------

Anything else?

> The only way you could limit that is if you did a static analysis
> and forbade any insertion of probes on any instruction preceeding
> a call that _may_ result in a process scheduling ... Surely you see
> this can't scale.

I don't see why that analysis is required.
We can simply suggest that user should avoid a call
instruction.

The problem is EIPs which is included with replacing
code on stack. So there is no problem when they don't
try to replace call instruction.


>>In addition, all CPU run on bypass code after int3 bypass
>>is created. (In another word, once int3 bypass would be set,
>>all CPU never push replacing instruction address on it's stack)
>>
>>So we need to take care of EIPs on current process of all CPUs
>>and interrupt stack. Now we are implementing this check code,
>>and we will provide soon.
> 
> But you have no way to figure out whether what you've found on the
> stack is an address to some piece of code or just some valid data ...

We are implementing two different way to check this.

First one:
Each interrupt handler push EIP on the stack to djprobe's
per cpu data structure before calling do_irq or something,
and pop EIP after returning.For checking safety,
djprobe look through this pushed EIPs.

djprobe can easily check EIPs which are included on stacks.

But we are afraid that upstream would not accept this
approach. So we are now trying another one.


Second one:
Simply looking through current stack and interruption stack.
djprobe may find the data that is same to an address to replace.
When it would happen, djprobe can easily postpone to replace
and wait for next check.

This implementation brings some delay to replace int 3 with
jmp. But probe code is still valid by kprobe and there is
no other side effect. Probe cost is same as kprobe.


>>I know that problem. Current djprobe's helper script show
>>disassemble code and prompt to avoid inserting a probe code
>>into such place.
>>
>>We may need to develop check function to avoid this problem,
>>but it will be an userland tools. We expect that translator
>>would provide these safety check, if possible.
> 
> So therefore, what this will do is, for each probe address candidate
> for an instruction less than 5 bytes, it will go through all of the
> kernel code to make sure that there are no references pointing to the
> next instruction(s) ... ? This after having checked on process
> stacks to make sure no one has a reference to those same addresses
> while somehow figuring out whether what's being looked at is not
> some data, but really a return address?

Currently we have no plan to limit djprobe not to use for
less than 5 bytes instruction. But when we would move to it,
djprobe will not provide any check on stack. There is no
problem when a stack has the same addresses to replace
if the candidate is more than 4 byte. Because a processor
can run jmp instruction instead of replaced code or int 3
instruction.

Satoshi