bfd: use less memory in string merging

[Michael Matz]

Hello,

On Wed, 8 Nov 2023, Jan Beulich wrote:

> > +  /* We allocate the ofsmap arrays in blocks of 2048 elements.
> > +     In case we have very many small input files/sections,
> > +     this might waste large amounts of memory, so reallocate these
> > +     arrays here to their true size.  */
> > +  amt = secinfo->noffsetmap + 1;
> > +  secinfo->map_ofs = bfd_realloc (secinfo->map_ofs,
> > +				  amt * sizeof(secinfo->map_ofs[0]));
> > +  BFD_ASSERT (secinfo->map_ofs);
> > +  secinfo->map = bfd_realloc (secinfo->map, amt * sizeof(secinfo->map[0]));
> > +  BFD_ASSERT (secinfo->map);
> 
> Re-use of the same block when shrinking isn't guaranteed, so depending
> on the underlying allocator this may actually add memory pressure (and
> the allocations may therefore also fail).

That's true, strictly speaking, but when this doesn't average out over 
thousands of blocks then it's such a low quality malloc(3) that it will 
have many other problems as well.  Certainly with the cases that lead me 
to the above (linking running nearly out of 32bit address space).  So I 
had that worry as well and rejected it.

> I think it would be nice to be independent of such an implementation 
> detail of the underlying library.

Yes.  But do you mean it as pre-requisite for the patch?  In that case 
I'll try something about a bucket allocator for the offsetmap blocks, 
though I think it's a bit on the extreme to work around lousy mallocs in 
current times.

> (It may also be worthwhile then to shrink the larger of the
> two arrays first. Otoh the comment ahead of mapofs_type already
> indicates that this type may need widening at some point.)

That is true nevertheless, so consider this changed in the patch.


Ciao,
Michael.