[PATCH] nptl: Disable THP on thread stack if it incurs in large RSS usage

[Adhemerval Zanella Netto]

On 16/05/23 12:38, Wilco Dijkstra wrote:
> Hi Adhemerval,
> 
>>> This still doesn't make sense since if _STACK_GROWS_DOWN, mem == guard, so
>>> this will always execute the madvise. 
> 
> And if !_STACK_GROWS_DOWN, we never execute the madvise. So I don't believe
> this is correct, even if it behaves like a nop in some cases.
> 
>> Yes, if THP is set to always this is exactly the idea of this patch since
>> afaiu the kernel might still back up the stack with large pages if the 
>> request a size is smaller than the default THP. 
> 
> If a mmap start/end range does not align to a huge page, you get small pages
> at the ends because a huge page does not fit.

Yeap, this is why Curpertino original testcase adds a way to force mmap
filling the VMA to triggers the THP allocated stack.

> 
>> It is only an issue if
>> the guard page address is not aligned to THP default size, which will
>> potentially trigger issues Cupertino has brought (since we do not prior
>> hand which is the mapping flags used on page used to fulfill the allocation). 
> 
> I don't see the claimed issue happen. What happens is that if you request
> huge pages, you get them. And that is what increases the RSS size.
> 
>>> As I mentioned, I couldn't find evidence that
>>> the claimed scenario of a huge page allocated, written to and then split due to the
>>> mprotect exists.
>>
>> I adapted Cupertino original test to allow specify both the thread stack
>> and guard size by command line.  Just:
> 
> The RSS size difference is not evidence of an issue - you asked for huge pages
> and you got them! I verified they are definitely huge pages by counting the TLB
> misses when accessing the stack.
> 
>>> So the real issue is that the current stack allocation code randomly (based on
>>> alignment from previous mmap calls) uses huge pages even for small stacks.
>>
>> Keep in mind this heuristic is only enabled if THP is set to 'always', meaning
>> the kernel will try to back *all* the stack with large pages.  The issue is
>> when the *guard* page is within a large page.
> 
> Why would that be an issue? In that case you can't get a large page.
> 
> The question is, under what circumstances are huge pages in stacks beneficial and
> in which cases are they not? If have a good answer to that, then we can automatically
> do the right thing without needing a tuning.
> 

Afaiu the issue is not whether huge page in stacks is beneficial, but rather 
when kernel will fall back to default pages anyway which will be just waste 
cycles.  Another option would be tune the default stack and guard size
to avoid this issue, but this might require some more heuristics to find a 
good spot to avoid too much VMA waste.