This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: tracking memory map changes


Jim Keniston wrote:
> On Fri, 2008-05-30 at 11:42 -0500, David Smith wrote:
>> One of the requirements of doing user-space probing is being able to
>> follow user-space program memory map changes.  I've been poking around
>> and I thought I'd share what I found and the possibilities of where to
>> go from here.
>>
>> I did all of the following on f8 and looking at 2.6.24.4 kernel source.
>>  I took a peek at the source for 2.6.26, but didn't see any drastic
>> differences in this area.
>>
>> Let me start by describing what is composed by a memory map.
> 
> ...
> 
> Yes, your description is consistent with my understanding.
> 
>> Frank, here are some initial questions.
>>
>> Q1: What information will the runtime need from each vm_area_struct?
>> I'd guess the path, vm_start, and vm_end at a minimum.
>>
>> Q2: Will the runtime want to know only about new text sections being
>> added or all sections?
> 
> It'd also be good to watch for vm areas going away.  I think Anil
> Keshavamurthy mentioned that this can happen for things like Java.
> Uprobes can theoretically get into trouble if you register a uprobe,
> then the underlying vma gets unmapped, then a new vma gets mapped in the
> same address range.  We should catch the munmamp/mremap and unregister
> any associated uprobes.

If CONFIG_PROFILING is on, sys_munmap() has a hook built into it that we
might be able to use.  There is a routine called profile_munmap() that
ends up using the blocking_notifier_chain kernel facility.  Of course
we'd have to see if we can do this without bothering profiling.  (I'm
not sure this will work at all, but it warrants further investigating.)

If that won't work, we'll have to fall back to something similar to what
we decide to do for mmap tracing.

>> So, where/how to track memory map changes?  Here are a few ideas:
>>
>> 1) Set a kretprobe on sys_mmap2()/do_mmap_pgoff()/mmap_region().  One
>> problem here is that kretprobes are limited in quantity.
> 
> Kretprobe_instances are preallocated per-kretprobe, so they're the
> limited quantity.  But quantity is a concern only if there are multiple
> instances of that particular probed function running concurrently.  We
> can play with maxactive to come up with a "safe" number... then triple
> that (or make it a -D option ;-)).
> 
> A problem with a k[ret]probe is that the handler can't block, so that
> limits what we can do.

Ah, OK.

>> 2) Set a kprobe on vm_stat_account().  This would require that the
>> kernel was configured with CONFIG_PROC_FS and that vm_stat_account() is
>> getting called in the correct place in all the kernels we're interested in.
>>
>> 3) Turn on utrace syscall return tracing for that thread and wait for
>> mmap calls to return.  This is probably the easiest route, but it forces
>> every syscall for that thread to go through the slow path.  A big
>> advantage here is that an all utrace solution wouldn't require any
>> debugging info for the kernel to be present on the system.
> 
> Yes.
> 
>> Does any have any better ideas or preferences here?
> 
> "mmap change event" is already on Roland's utrace TODO list.  That'd be
> my preference.  It's be lightweight, and you could block if necessary to
> do your magic when notified.  You obviously know the terrain well, so
> maybe you could funnel that work into the utrace enhancement.

You are right, I've forgotten this one.  This would certainly be better
than the other methods listed.  The biggest problem with this one is
that it would require kernel changes (which would probably mean we
couldn't use it with older kernels).

>> In all of the above methods the code won't know what was added, just
>> that a new vm_area_struct might exist, so I'll have to figure out a way
>> to track changes.
>>
>> Finally, the shortest path to something somewhat useful would be to
>> first work on providing notification of existing vm_area_structs.  This
>> might help move user-space tracing along while I work on the harder
>> problem of tracking memory map changes.  Providing notification of
>> existing vm_area_structs might allow attaching to an existing thread
>> (which already has all its shared libraries loaded and doesn't call
>> dlopen()) and being able to figure out the right address to probe.
>>
> 
> You don't explicitly say so, but I infer from this that you're making
> progress on probing shlibs.  That'll be huge.

That's the goal.  I'm just trying to figure out how to get there.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]