thread heap leak?

[Carlos O'Donell]

On 3/22/19 1:39 PM, David Muse wrote:
> On Fri, 22 Mar 2019 13:03:51 -0400 "Carlos O'Donell"
> <codonell@redhat.com> wrote:
> 
>> On 3/22/19 10:53 AM, David Muse wrote:
>>> We have this little server program that listens for client 
>>> connections on an inet socket.  When it receives a connection,
>>> it pthread_create()'s a detached thread to handle the request and
>>> the main thread waits for more client connections.
>> 
>> The stack for the detached thread is recoverable only if the
>> thread exits, at which point it calls 	__free_tcb () (only if
>> detached, because otherwise only pthread_join() does this
>> recovery), enqueues the stack onto the free list and leaves it
>> there. Subsequent thread creation attemps to reuse the stack, but
>> must wait for the kernel to clear the registered tid memory
>> (CLONE_CHILD_CLEARTID) to mark the stack reusable (FREE_P). The
>> total size of the thread stack cache should only be ~40MiB, and new
>> stack creation triggers an automatic check to trim this cache back
>> down to 40MiB.
>> 
>> Have you been able to reduce this to a smaller test case?
>> 
>> The anonymous mappings might indeed be thread stacks, but you'd 
>> have to verify that yourself, and you can do that by asking the 
>> thread for the stack information and printing that.
>> 
>> Are you using Amazon Linux or a known distro upstream?
> 
> I did comment out lots of features in the program to narrow down the
> problem, but I haven't been able to reproduce the leak in a compact
> test case.  I've been working on that though.

Good luck :-)

> How do I get the stack information from the thread?  I know that I
> can do pthread_attr_getstacksize for the size, but what else can I
> get?

You can use pthread_getattr_np() to take a snapshot of the thread's
attributes and look at those, that will get you the information about
stack address location and size.

You should try using something like systemtap to put tap points on
your program and observe the behaviour of the detached threads.


> We're using OpenSUSE Leap 42.3.  I've seen the problem directly
> there.  I've gotten reports that it also occurs on RHEL 7, but I
> can't confirm that.

Andreas,

Have you seen reports like this?

-- 
Cheers,
Carlos.