Re: task_finder callback (and engine?) for every uprobe

[David Smith <dsmith at redhat dot com>]

Frank Ch. Eigler wrote:
> Hi -
> 
> On Tue, Dec 16, 2008 at 10:48:17AM -0600, David Smith wrote:
>> Jim Keniston wrote:
>>> Investigating PR 7082 & 7092 -- and in particular, instrumenting
>>> task_finder's callbacks -- has reminded me of something we discussed
>>> very briefly earlier: if we have N uprobes in a process, then every time
>>> something happens in that process that requires a task_finder callback,
>>> we get N (calls to) callbacks.  E.g., when I probe bash (2170 uprobes),
>>> callbacks happen in blocks of about 2170.  Just starting up one instance
>>> of bash yields about 101,900 task_finder callbacks on 2.6.27.  [...]
>> [...]
>> (5) uprobe probes on "*" on a specific bash shell
>> #  stap -DDEBUG_TASK_FINDER -v -e 'probe process("bash").function("*") {
>> }' -wc bash
>> total attach count: 70070, attach count: 916, inuse count: 1
>> callback count: 139173, vm_callback count: 0
>>
>> Ouch.  According to the pass 2 output, there are 2694 probes.
>> [...]
>> So, based on all the above, I don't think this is a task_finder problem,
>> but a stap translator problem.  The task_finder can't know when you
>> really need a callback and when you don't, but the translator could
>> figure that out.  Jim/Frank, got any different opinions here?
> 
> Looking at the stap -p3 output, it does request a separate
> stap_task_finder_target for each uprobe.  However, all those requests
> are identical (in terms of callback function pointers and
> pathname/etc.).
> 
> The task finder could merge all those requests into one utrace engine
> attachment.  Or the translator could put in a corresponding level of
> indirection.  Either can be made to work.  (The former has the
> advantage of editing C code rather than editing a C code generator,
> and its greater general usefulness.)

Yep, it certainly could be done either way.

There are pros and cons to either way.  The biggest advantage to putting
this logic in the task_finder is that it would be in 1 place instead of
3 (utrace/uprobes/itrace probes in the translator).  The biggest
advantage to putting this logic is the translator is that we'd be doing
the processing (combining probes) at translation time instead of in the
kernel at run time.

To me, the latter reason outweighs the former.  Perhaps we can come up
with some common code that utrace/uprobes/itrace probes can use in the
translator to have the best of both worlds.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)