LD_AUDIT: Not enough space in static TLS block

[Florian Weimer]

* Florian Weimer:

> * Jonathon Anderson:
>
>> Hello all,
>>
>> We (the HPCToolkit team) have encountered another critical LD_AUDIT
>> bug. When LD_AUDIT is specified, the allocation of the static TLS
>> block does not account for the TLS requirements of executable
>> dependencies or of the auditors themselves. If:
>>  - an executable accesses a thread-local variable in a linked library
>> with sufficiently large TLS requirements, or
>>  - an auditor itself uses sufficiently large TLS and optimizes access
>> with `-ftls-model=initial-exec`,
>>
>> then the process or auditor will fail with the error "cannot allocate
>> memory in static TLS block."
>
> We have a tunable that can be used as a workaround.  Your reproducer
> passes for me with our 2.28 backport (glibc-2.28-164.el8) if I run it
> like this:
>
>   GLIBC_TUNABLES=glibc.rtld.optional_static_tls=4000 make
>
> The best we can do in the short term would be an increase of the default
> limit.  On 64-bit platforms, defaulting to a dozen or so kilobytes per
> thread should not be a problem as far as virtual address space
> consumption is concerned.  We can also add an additional reservation of
> similar size for every auditor that is loaded, to compensate for the
> lack of auto-tuning of the TLS allocation size in auditing mode.
>
> The fundamental issue is that there is always going to be a hard limit
> for initial-exec TLS.  Initial-exec TLS requires a fixed offset from the
> thread pointer, and we cannot relocate TLS variables because they are
> ordinary C objects with an observable address.  There are some other
> things we can try to improve auto-tuning, but in the end, there is
> always going to be a fixed-size reserved area dedicated to initial-exec
> TLS set up at process startup, and with dlopen, that might not be enough
> even without any auditor use.

Jonathon,

does setting the environment variable work for you?

Thanks,
Florian