Re: [RFC][PATCH 2/2] uprobes: single-step out of line

[Jim Keniston <jkenisto at us dot ibm dot com>]

On Fri, 2007-05-04 at 21:09 -0400, Ernie Petrides wrote:
> On Friday, 20-Apr-2007 at 15:09 PDT, Jim Keniston wrote:
> 
> > This patch enhances uprobes to use "single-stepping out of line"
> > (SSOL) to avoid probe misses in multithreaded applications.  SSOL
also
> > reduces probe overhead by 25-30%.
> 
> 
> Hi, Jim.  I just had one question about this (mainly because I'm not
> able to work through all the nitty gritty details of the patch).
> 
> Is there a way to avoid adding the "uprobe_ssol_area" struct into the
> "mm_struct"?  If so, the uprobes module could be easily back-ported to
> kABI-frozen distros of the Linux kernel.  If the "mm_struct" ends up
> getting changed, that changes the "task_struct" layout, thus breaking
> binary compatibility with 3rd-party kernel modules.  (We're not
allowed
> to do this in RHEL distros.)
> 
> Cheers.  -ernie

Hmmm.  It does indeed change the layout of struct mm_struct.  I don't
see how it changes the layout of task_struct, since task_struct
contains only pointers to mm_structs.  But changing mm_struct itself
is bad, right?

An obvious alternative is for uprobes to maintain this pointer
in one of its own data structures.  Currently, when the last uprobe
for a process is unregistered, we discard the uprobe_process and
uprobe_tasks, and the only thing that remains is the pointer to
the uprobe_ssol_area (in mm_context).  We need to remember that
pointer in case the process is probed again -- we want to reuse the
vma.  Two sub-alternatives occur to me:

1a. Once a process is probed, keep the uprobe_process (and
uprobe_tasks and utrace engines) around 'til the process dies.
(We need the utrace engines because they're what tell us when the
threads die.)

1b. Store just the uprobe_ssol_area pointer in a hash table.
Every time a process exits, check that hash table and free up
the little pointer object if there is one for that process.
I'm not sure where I'd put that check -- maybe __mmdrop()?
I don't know my way around do_exit() very well.  And I wonder
if people will object to adding more code to the exit path.

Another alternative is to free up the vma every time we free
the uprobe_process.  I don't like that idea much.

One thing to keep in mind is that with uprobes patch #4 (was patch
#3 in Q1 2007 -- hasn't been reposted recently), uprobe handlers
can [un]register probes, so the number of registered probes for
a process can bounce between zero and non-zero rather frequently.
This recommends approach #1a.

Jim