This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: RFC: api to walk memory allocations


On 10/30/13, 6:58 PM, Will Newton wrote:
On 30 October 2013 17:41, Rich Felker <dalias@aerifal.cx> wrote:
On Wed, Oct 30, 2013 at 04:30:59PM -0600, David Ahern wrote:
We are exploring options for debugging memory corruption and leaks.
One of the ideas involves walking allocations and extending glibc
with a generic hook for that. Before we get too far down this road I
wanted to get thoughts from the glibc maintainers on the idea and
whether such an extension would be accepted into glibc.

The API we have discussed is:
[...]

The idea here is to have something that can be used within gdb as
well as something that could be invoked runtime for a live dump. Any
feedback on the idea would be appreciated.

My view is that this kind of invasive, heavy debugging interface does
not belong in glibc or anywhere it's introspective, but as part of the
debugger. Valgrind already does a great job with this kind of thing,
and gdb could do some similar work just by breakpointing malloc and
free and recording the calls. However, that's just my opinion and
certainly not authoritative for glibc.

I agree. This type of thing has the habit of becoming an API and then
has to be supported which places restrictions on how malloc can be
implemented.

I was hoping this would be seen in the same light as mcheck and mtrace - but along the lines of something you can use in a production environment.


BTW, I understand that you may want to be able to do this kind of
debugging on a core dump or a running application that was not
originally started in the debugger (in which case the gdb recording
approach would not work), but I don't think committing glibc to
maintaining this degree of meta-data overhead when not debugging is a
reasonable long-term constraint. Even if it's easy to implement with
the current malloc implementation (I doubt it is; generally, only free
chunks, not allocated chunks, are tracked in an efficient malloc
implementation), requiring the tracking would place constraints on
what sorts of optimizations could be made in the future.

SystemTap could be another option for implementing this type of thing.

I am talking about production systems that are up and running for months to years. You do not run gdb or valgrind or any other debugger in these scenarios - and that includes SystemTap, perf-probes, etc - for a process that can have millions of allocations.

One scenario here is a customer suspects a memory leak in a process and runs a CLI command that essentially walks the memory data structures. With the callback we can run a sanity checker or what have you. There's more to it from an infra perspective, but that gets into ap design. glibc wise it is a matter of walking glibc data structures and allowing aps to have a means of sanity checking.

We have discussed other options for what we need. I would prefer something builtin that leverages the existing data structures. No sense adding layers, complexity and overhead. We are not the only ones in need of a memory tracking, leak, corruption capability.

David


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]