Re: [gold][patch] Reduce heap usage for string merge sections

[Cary Coutant <ccoutant at google dot com>]

> This time with the patch attached.

OK, this time with the *right* patch attached. Sorry!

> On Tue, Jul 20, 2010 at 4:09 PM, Cary Coutant <ccoutant@google.com> wrote:
>> I used tcmalloc to profile the heap usage of gold while building a
>> large binary compiled with -O2 and -g, and found that the
>> Merged_strings structures were responsible for just over 50% of the
>> total heap used by gold. That structure has 6 fields: a pointer to the
>> object, the input section index, the input offset, a pointer to the
>> string itself, the length of the string, and the stringpool key. The
>> string field was never used, so I removed it. The pointer to the
>> object and the input section index were common across fairly large
>> lists of strings, so I separated the one list per output section into
>> separate lists for each (object, shndx), and removed those two fields
>> from the Merged_string struct. The length field was easily recomputed
>> by subtracting adjacent input offsets, so I removed it, too. The
>> result is a 67% reduction in the size of the Merged_string structure,
>> and the net effect was to reduce the total heap used for my benchmark
>> from 3.45 GiB to 2.56 GiB (25% less heap).
>>
>> The Object_merge_map structure is now responsible for more heap than
>> anything else; I'll be taking a look at reducing its memory footprint
>> next.
>>
>> Tested on x86_64 Linux. OK?
>>
>> -cary
>>
>>        * merge.h (Output_merge_string::Merged_string): Remove object, shndx,
>>        string, and length fields.
>>        (Output_merge_string::Merged_strings_list): New type.
>>        (Output_merge_string::Merged_strings_lists): New typedef.
>>        (Output_merge_string): Replace merged_strings_ with
>>        merged_strings_lists_.
>>        * merge.cc (Output_merge_string::do_add_input_section): Allocate new
>>        Merged_strings_list per input object and section.  Don't store pointer
>>        to the string.  Don't store length with each merged string entry.
>>        (Output_merge_string::finalize_merged_data): Loop over list of merged
>>        strings lists.  Recompute length of each merged string.
>>
>

Attachment: gold-merge-string-patch.txt
Description: Text document