Re: RFC: api to walk memory allocations

[David Ahern <dsahern at gmail dot com>]

On 10/30/13, 9:31 PM, Rich Felker wrote:
> On Wed, Oct 30, 2013 at 09:09:02PM -0600, David Ahern wrote:
> > One scenario here is a customer suspects a memory leak in a process
> > and runs a CLI command that essentially walks the memory data
> > structures. With the callback we can run a sanity checker or what
> > have you. There's more to it from an infra perspective, but that
> > gets into ap design. glibc wise it is a matter of walking glibc data
> > structures and allowing aps to have a means of sanity checking.
>
> I'm not clear how much that really buys you. To debug the leak you
> need a lot more information about the owners of the allocations, not
> just the map of all existing allocations.

Yes, there is more to the design, but it is irrelevant to the glibc 
request so I did not include it. We have infrastructure to track each 
malloc/free. It is fairly trivial to hook malloc/free and add a 
footer/header to each allocation. The meta-data here contains whatever 
we feel is sufficient to track the source of the allocations. The part 
up for debate is efficiently walking the allocations without maintaining 
yet another data structure to do it.

> The problem with "leveraging existing data structures" is that it
> locks you into the current existing data structures. Building as a
> separate layer, or separate malloc implementation you could link in,
> may cost more for users of this feature, but avoids the perpetual cost
> for everyone else who doesn't need it.

The point of building something into glibc is that our code does not 
have to have knowledge of malloc internals. I can be rather dense at 
times, so can you elaborate on how a memwalk api restricts the malloc 
implementation? The whole point here is that nothing about the internal 
implementation is exposed. The memwalk API is simply pointer, length and 
a callback into the app. None of that involves the app knowing anything 
about how malloc works.

David