This is the mail archive of the
mailing list for the glibc project.
Re: static TLS exhausted on ppc64le
- From: Szabolcs Nagy <Szabolcs dot Nagy at arm dot com>
- To: Rich Felker <dalias at libc dot org>, Florian Weimer <fweimer at redhat dot com>
- Cc: nd <nd at arm dot com>, Dan Horák <dan at danny dot cz>, "libc-alpha at sourceware dot org" <libc-alpha at sourceware dot org>
- Date: Mon, 30 Sep 2019 16:13:07 +0000
- Subject: Re: static TLS exhausted on ppc64le
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YoVCMdHFXlshNfw3btK/0B12gJ++mc+efC068I5Ovmk=; b=N+DIsPo5Ya7ABbTFQbAu8tJBdj9BVvnCPP0IZVG8TpwaUijC3djc0eO4Qv8Hzvl2aOtbh+ZX0qW1A3q8bslJJsUWWPqNl2tDU6Sf059jd5J2oIbnNbXpZfAYdGCZaKx7Emv2X82wRdGzEJh6uIefRTbM+h26V58qY6nQGFrnP4jAA6achxhWoKuagQN6matAVmN+M+7xBKHRQZBXGFGTrabBGHxJxWxpZdkwVy3iviG2KoY+qXyTktOqU3IJ06yeIrnHXTVd7gS3BVTv9+PdGQqbRN55bUGFe1dxHyQ9eMeFwx0Ksef233KNrBHPXeuUtBwDHaFH6KgGtjMDMw/+MQ==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dQgCiTewDInLni2j6ZFK3aPt9bTGwAX5hxt+njqisoHrN0I0PRz6uG8p34yqokLer9A5meVKqqSeN1GCEg9jToxWd6J839rlkswPa1Z+Pr2KJGXn97M8Q0+9EA+uneJ9cjgJnGL21CCH16KeTA8bi8hS9n5rsNJccjto5yT/RevC/iKu8lYqoaV//auQ68MloqHlIqUv8E1b31oE5frNBXcPGFKVw8wuDzoC/iwx68Nvl/ZsfDCQA2PJPlEXTXqvKqF7C/SYUitNAvN+WC+PUdhNiBwYNp75wKmqCm6momJ7dCMRZkV4ciRR1mm6c5i5qGzUyQAhbDlb3Yx/t1hjRw==
- Original-authentication-results: spf=none (sender IP is ) smtp.mailfrom=Szabolcs dot Nagy at arm dot com;
- References: <email@example.com> <firstname.lastname@example.org> <email@example.com> <20190930160631.GP9017@brightrain.aerifal.cx>
On 30/09/2019 17:06, Rich Felker wrote:
> On Mon, Sep 30, 2019 at 05:47:29PM +0200, Florian Weimer wrote:
>> * Szabolcs Nagy:
>>> On 30/09/2019 15:02, Dan Horák wrote:
>>>> I would like to open a problem we have already met twice in Fedora. the
>>>> symptom is
>>>> "/lib64/libgomp.so.1: cannot allocate memory in static TLS block"
>>>> usually when loading a lot of libraries/modules into a Python
>>>> application. It happened on ppc64le and also on aarch64 systems.
>>>> We have 2 reports in Fedora bugzilla about with more details.
>>>> We have already discussed that briefly with Florian and other members of
>>>> the Red Hat toolchain team, but outcome was in form of a recommendation
>>>> to reduce the usage of "static TLS" objects in the individual libraries.
>>>> But the open question still is - is there a fix for the TLS space
>>>> exhaustion? I believe it can easily become a more serious problem soon.
>>> (a workaround is preloading the problematic libs at startup time)
>>> i think it's a bug in libgomp.so.1, gcc should not build
>>> broken dsos by default (unless it can ensure they are never
>>> loaded dynamically):
>> I like the simplicity of initial-exec TLS.
> I wouldn't really characterize it as simplicity. It's a trade of
> complex (at least to the user) constraints on whether or not it works
> for some simplicity of implementation.
> I guess for glibc at present, there's a lot more complexity to dynamic
> models because of lazy allocation and installation and generation
> counters, and these interact with AS-safety and failsafety in
> undesirable ways. I'd like to see that fixed but I know it's a big
>> I think there was a change on POWER to use the static TLS reservation
>> for dynamic TLS, as an optimization. Obviously, that's going to hurt
>> those cases where a library with initial-exec TLS is loaded late, even
>> if the static TLS reservation would ordinarily be large enough.
> Was that because of the PLT-stub hack on powerpc done in lieu of
> tlsdesc? That should really be abandoned entirely IMO, since it
> *doesn't* give you any of the benefit of tlsdesc -- the whole point is
> not the short code path but avoiding register spills for the standard
> ABI call to __tls_get_addr, and the powerpc hack doesn't let you avoid
> them. Real tlsdesc should be added to powerpc.
the problem is TRY_STATIC_TLS (defined in dynamic-link.h)
when it is used for dynamic tls (on targets where that's
possible: tlsdesc or ppc tls opt hack) it will eat the
preallocated static tls. (that's why this affects aarch64
i think that logic can be easily changed so the preallocated
tls area is not used for normal dynamically loaded dsos
(assuming the intention of the prealloc tls is purely to
support dsos with initial-exec tls), that's less optimal for
the common dynamic tls use-case, but makes libgomp etc work.