Re: pthread wastes memory with mlockall(MCL_FUTURE)

[Rich Felker <dalias at libc dot org>]

On Fri, Sep 18, 2015 at 05:38:42PM +0100, Balazs Kezes wrote:
> On 2015-09-18 10:38 -0400, Rich Felker wrote:
> > I would say it's a kernel bug for PROT_NONE pages to actually occupy
> > resources when locked, if they actually do?
> 
> It could make sense that you have some pages with some data in them but
> in a later stage you remove the permissions to trap data accesses. I
> think some debuggers (e.g. watchpoints) or some cache simulators work
> this way.

I'm talking about new PROT_NONE pages. The kernel certainly accounts
for them differently as commit charge. New PROT_NONE pages consume no
commit charge. Anonymous pages with data in them, which would become
available again if you mprotect them readable, do consume commit
charge. (For this reason, you have to mmap MAP_FIXED+PROT_NONE to
uncommit memory rather than just using mprotect PROT_NONE, even if you
already used madvise MADV_DONTNEED on it.)

> > How did you test/measure this?
> 
> There's a pthread_attr_setguardsize function. Basically I've set it to
> large (to expand the effect of this behavior) and then created a bunch
> of threads and checked what happens. I could try to create a simple
> repro if needed.

But you were able to measure them using physical resources? Or might
it possibly just be bad accounting. Utilities like 'top' are notorious
for misrepresenting the memory usage of a process.

Rich