Already reported on the glibc list [1] but creating a ticket here for tracking purposes. This is currently breaking ffmpeg based apps [2] [1] https://sourceware.org/pipermail/libc-alpha/2023-August/150653.html [2] https://bugs.archlinux.org/task/79300
Patch posted: [PATCH] malloc: Enable merging of remainders in memalign (bug 30723) <https://inbox.sourceware.org/libc-alpha/877cq4yrpg.fsf@oldenburg.str.redhat.com/T/> We may need additional heuristics to bound the free list scanning.
Hey thanks for the patch! Unfortunately multiple testers in the downstream Arch ticket are still getting 100% CPU when using the "mpv" app. Please see this comment [1] for a supposed reproducer (I haven't tested it yet...my role is downstream bug wrangler, so I'm just the message relay here currently..) [1] https://bugs.archlinux.org/task/79300#comment220881
Latest patch [1] on top of the previous one appears to have done the trick. Thanks so much for all your efforts. We will get this out to users once it hits the tree (and hopefully the 2.38 branch). [1] https://inbox.sourceware.org/libc-alpha/87pm3uajev.fsf@oldenburg.str.redhat.com/
First part committed: commit 542b1105852568c3ebc712225ae78b8c8ba31a78 Author: Florian Weimer <fweimer@redhat.com> Date: Fri Aug 11 11:18:17 2023 +0200 malloc: Enable merging of remainders in memalign (bug 30723) Previously, calling _int_free from _int_memalign could put remainders into the tcache or into fastbins, where they are invisible to the low-level allocator. This results in missed merge opportunities because once these freed chunks become available to the low-level allocator, further memalign allocations (even of the same size are) likely obstructing merges. Furthermore, during forwards merging in _int_memalign, do not completely give up when the remainder is too small to serve as a chunk on its own. We can still give it back if it can be merged with the following unused chunk. This makes it more likely that memalign calls in a loop achieve a compact memory layout, independently of initial heap layout. Drop some useless (unsigned long) casts along the way, and tweak the style to more closely match GNU on changed lines. Reviewed-by: DJ Delorie <dj@redhat.com> Second part is still under review, I need to send a v2.
Second part committed: commit 0dc7fc1cf094406a138e4d1bcf9553e59edcf89d Author: Florian Weimer <fweimer@redhat.com> Date: Thu Aug 10 19:36:56 2023 +0200 malloc: Remove bin scanning from memalign (bug 30723) On the test workload (mpv --cache=yes with VP9 video decoding), the bin scanning has a very poor success rate (less than 2%). The tcache scanning has about 50% success rate, so keep that. Update comments in malloc/tst-memalign-2 to indicate the purpose of the tests. Even with the scanning removed, the additional merging opportunities since commit 542b1105852568c3ebc712225ae78b ("malloc: Enable merging of remainders in memalign (bug 30723)") are sufficient to pass the existing large bins test. Remove leftover variables from _int_free from refactoring in the same commit. Reviewed-by: DJ Delorie <dj@redhat.com> I will handle the 2.38 backport after a brief delay.
The glibc 2.38 release branch has been fixed.