Re: boosting with preemptable kernel

[Masami Hiramatsu <masami dot hiramatsu dot pt at hitachi dot com>]

Hi Quentin,

Quentin Barnes wrote:
> I disagree that an such approach increases overhead by any
> significant amount and is only useful for booster code, but more on
> this later down in the mail at the end.
>
> I really disagree with the garbage collector approach.  On some
> of our projects we use the Linux kernel in a soft real-time
> environment.  A GC introduces sporadic indefinite postponement
> problems.  It doesn't matter at all how often it runs -- once is too
> much.  The basic reason for having a preemptive kernel is to reduce
> indefinite postponement.  The GC is most disruptive on preemptive
> kernels working against the very reason for using the preemptive
> feature to begin with.  (It's so disruptive on preemptive kernels
> because it calls freeze_processes() affecting the entire system.  On
> non-preemptive kernels it uses the passive synchronize_sched().)

Indeed.

> Aside from RT related impact issues the GC causes, it also degrades
> poorly on SMP systems.  It causes the entire system across all CPUs
> to be forced to stop executing and all held into an idle state
> simultaneously.  The reason for having multiple CPUs is so that a
> thread tying up one CPU doesn't impact the performance (much) of
> the rest of the system.  Now we have a single thread being able to
> impact the performance across all CPUs system-wide simultaneously.
> (If my understanding of the GC implementation is wrong, please
> correct me.  It's only from my limited understanding of reading and
> following the code, not from using it.)

I think you are a bit misreading the GC implementation.
First, this GC is never invoked when the kprobe is hit.
This GC may be invoked when you register/unregister a kprobe.
I think these operations are not frequently done and it will
be done when the module is loading/unloading.

Next, this GC will be invoked if there ARE some garbage(dirty)
slots when you get/free an insn slot. Thus, if your kprobe clean
its slot up before release it, your kprobe NEVER invokes the GC.

> What I would recommend is that the GC and its hooks be able to be
> conditionally enabled or disabled with an ifdef, probably in the
> arch kprobes.h file like insn slot and kretprobes are now.  That way
> for architectures that don't need it by having their own alternate
> approach can choose not to use it.  Would that be a reasonable
> suggestion?

Currently, only the slots used by boosted kprobes are dirty.
So, on the ARM archtecture, if you'd like disable the GC,
you just call free_insn_slot with dirty=0, as below.
 free_insn_slot(insn_slot, 0);

on i386, you can prohibit boosting kprobes by specifying a void
post_handler to the kprobes. And then, the GC will not work any more.

If you feel a strong need for disabling GC at compiling,
I can add the compile flag for disabling GC. Or,
could you write the trampoline routine and the
compile flag for switching the trampoline and the GC?

> About your last bullet mentioning djprobes, the ARM implementation
> of kprobes I'm working on does have support for kretprobes and
> jprobes, but not djprobes.  I have read your post from Oct '05 on
> djprobes and understand it in a general way, but haven't yet wrapped
> my mind fully around it.  Could you explain why such an approach
> doesn't work for djprobes on x86?  Do you imagine that assertion
> would hold for other non-x86 architectures as well?

Djprobe has to rewrite multiple instructions on i386 (CISC), because
the size of the long jump instruction is bigger than many instructions.
Before rewriting those instructions, we must ensure no other processes
running/sleeping on the target instructions which will be rewritten by
the long jump. This case can not be helped by the trampoline routine.

Best regards,

-- 
Masami HIRAMATSU
Linux Technology Center
Hitachi, Ltd., Systems Development Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com