alloca avoidance patches

[Paul Eggert]

On 06/19/2017 09:47 AM, Joseph Myers wrote:
> we need a clear definition of what stack frame size
> glibc can assume is safe (so that we can aim to eliminate alloca and VLAs
> unless the compiler can see they are bounded, and use -Wstack-usage= for
> building glibc to make sure no function uses too much stack).

For what it's worth, Gnulib uses the magic number 4032. That is, arrays 
containing more than 4032 bytes are supposed to be allocated on the heap 
rather than on the stack via alloca. This number was chosen to work on 
all platforms Gnulib is intended to be portable to; we didn't bother to 
tailor the number to each platform. The basic assumptions in Gnulib are:

* Every stack is guarded by at least 4096 bytes' worth of inaccessible 
pages; the 4032 is because we subtract a few bytes for local variables.

* There is at most one such array per stack frame.

* The compiler does not coalesce stack frames for functions that declare 
large locals. This is a bit tricky, as a function that allocates a local 
buffer via 'char buffer[4000];' is not safe if it calls another function 
that allocates a similar local buffer, because the call could be 
inlined. In theory I suppose this could be a problem even with alloca, 
but in practice I don't recall seeing it happen with alloca.

PS. My first reaction was to ask "how's -fstack-check going?" but I see 
Jeff Law beat me to it.