This is the mail archive of the
mailing list for the glibc project.
Re: RFC: api to walk memory allocations
- From: David Ahern <dsahern at gmail dot com>
- To: Will Newton <will dot newton at linaro dot org>, Rich Felker <dalias at aerifal dot cx>
- Cc: libc-alpha <libc-alpha at sourceware dot org>
- Date: Wed, 30 Oct 2013 21:09:02 -0600
- Subject: Re: RFC: api to walk memory allocations
- Authentication-results: sourceware.org; auth=none
- References: <527188A3 dot 4000305 at gmail dot com> <20131031004120 dot GF20515 at brightrain dot aerifal dot cx> <CANu=Dmj-8iEPgfA69yNmPNSxOXvX5u_SNSdUrNMJeipCG+Apqg at mail dot gmail dot com>
On 10/30/13, 6:58 PM, Will Newton wrote:
On 30 October 2013 17:41, Rich Felker <email@example.com> wrote:
On Wed, Oct 30, 2013 at 04:30:59PM -0600, David Ahern wrote:
We are exploring options for debugging memory corruption and leaks.
One of the ideas involves walking allocations and extending glibc
with a generic hook for that. Before we get too far down this road I
wanted to get thoughts from the glibc maintainers on the idea and
whether such an extension would be accepted into glibc.
The API we have discussed is:
The idea here is to have something that can be used within gdb as
well as something that could be invoked runtime for a live dump. Any
feedback on the idea would be appreciated.
My view is that this kind of invasive, heavy debugging interface does
not belong in glibc or anywhere it's introspective, but as part of the
debugger. Valgrind already does a great job with this kind of thing,
and gdb could do some similar work just by breakpointing malloc and
free and recording the calls. However, that's just my opinion and
certainly not authoritative for glibc.
I agree. This type of thing has the habit of becoming an API and then
has to be supported which places restrictions on how malloc can be
I was hoping this would be seen in the same light as mcheck and mtrace -
but along the lines of something you can use in a production environment.
BTW, I understand that you may want to be able to do this kind of
debugging on a core dump or a running application that was not
originally started in the debugger (in which case the gdb recording
approach would not work), but I don't think committing glibc to
maintaining this degree of meta-data overhead when not debugging is a
reasonable long-term constraint. Even if it's easy to implement with
the current malloc implementation (I doubt it is; generally, only free
chunks, not allocated chunks, are tracked in an efficient malloc
implementation), requiring the tracking would place constraints on
what sorts of optimizations could be made in the future.
SystemTap could be another option for implementing this type of thing.
I am talking about production systems that are up and running for months
to years. You do not run gdb or valgrind or any other debugger in these
scenarios - and that includes SystemTap, perf-probes, etc - for a
process that can have millions of allocations.
One scenario here is a customer suspects a memory leak in a process and
runs a CLI command that essentially walks the memory data structures.
With the callback we can run a sanity checker or what have you. There's
more to it from an infra perspective, but that gets into ap design.
glibc wise it is a matter of walking glibc data structures and allowing
aps to have a means of sanity checking.
We have discussed other options for what we need. I would prefer
something builtin that leverages the existing data structures. No sense
adding layers, complexity and overhead. We are not the only ones in need
of a memory tracking, leak, corruption capability.