[PATCH] bfd: Grow in-memory BFD buffers geometrically
Jan Beulich
jbeulich@suse.com
Fri Nov 6 09:46:56 GMT 2026
On 02.11.2026 23:59, Oleg Tolmatcev wrote:
> memory_bwrite and memory_bseek grew the buffer in 128 byte steps, so
> building a large in-memory BFD out of many small writes reallocated and
> copied the whole buffer over and over. This made writing PDB streams
> quadratic: linking a 235 MB executable with --pdb spent more than half
> of its time in realloc.
>
> Round the allocation up to a power of two instead.
Feels like moving from one (almost) extreme to another. Now a 256Mb + 1
byte executable will require an allocation of half a Gb to succeed. So
far, when a rounded-up allocation failed, we can assume one attempted
with the original size would also fail. This isn't true anymore. Hence
minimally I think we need fallback logic to retry with less of an
increment if an allocation failed.
That'll still leave potentially excessive memory use, so the above may
not be sufficient to limit unwanted effects. Going purely geometrically
may simply be too steep a growth.
Tangentially: I'm puzzled by memory_bseek() doing any allocation at all.
As long as there was no write past the new trailing zeroes, surely there
is pretty little need for all that zeroed memory? (Yes, avoiding that
would require some extra bookkeeping.)
Jan
More information about the Binutils
mailing list