Bug 26406

Summary: Extreme assembling time regression with 2.35 and master
Product: binutils Reporter: franz.sirl
Component: gasAssignee: Nick Clifton <nickc>
Status: RESOLVED FIXED    
Severity: normal CC: amodra, franz.sirl, mail, martin.liska, nickc, romain.geissler
Priority: P2    
Version: 2.35   
Target Milestone: ---   
Host: x86_64-linux-gnu Target: x86_64-linux-gnu
Build: x86_64-linux-gnu Last reconfirmed:
Attachments: Proposed patch

Description franz.sirl 2020-08-19 12:41:24 UTC
Hi,

I just stumbled over an extreme slowdown in assembler performance. On a 265 MByte .s file (gcc10 assembly output of a 27000 line C source compiled with -O2 -fno-inline + debug + ASAN + UBSAN) the assembler of 2.34 takes ~49-59sec, but the 2.35 assembler takes ~65-74min. Both use about 3.6 GBytes of memory. That's quite a performance regression.

fsirl@ds1:~> time as-2.35 --64 -o sample.o sample.s

real    74m11.921s
user    73m45.203s
sys     0m13.132s
fsirl@ds1:~> time as-2.34 --64 -o sample.o sample.s

real    0m49.454s
user    0m46.331s
sys     0m2.623s

Most of the time seems to be spent in _bfd_elf_write_secondary_reloc_section().

The slowdown started with this commit:

commit a8e14f4cc2badfcf959f5e2cc57a941dc43f72d4
Author: Nick Clifton <nickc@redhat.com>
Date:   Thu Mar 5 15:47:15 2020 +0000

    Add support for ELF files which contain multiple reloc sections which all target the same section.

Unfortunately I cannot share the testcase, but I can do any requested debugging/testing.
Comment 1 Alan Modra 2020-08-19 13:10:36 UTC
We're quadratic in number of sections.

_bfd_elf_write_object_contents
  bfd_map_over_sections (abfd, bed->s->write_relocs, &failed);

elf_write_relocs:
  if (!bed->write_secondary_relocs (abfd, sec))

_bfd_elf_write_secondary_reloc_section:
  for (relsec = abfd->sections; relsec != NULL; relsec = relsec->next)
Comment 2 franz.sirl 2020-08-19 14:44:05 UTC
Well, that would explain it:

fsirl@ds1:~> grep '\.section' ~/sample.s |sort -u|wc -l
294085

The testcase was compiled with -ffunction-sections -fdata-sections.
Comment 3 Nick Clifton 2020-08-19 16:57:49 UTC
Created attachment 12781 [details]
Proposed patch

Hi Franz,

  Please could you try out this patch and let me know if it works for you.

  It will only improve the performance in the case of linking files which
  do not contain secondary relocs, but I am assuming that this applies to 
  you.

  A proper fix, involving chaining the secondary reloc sections together
  should be written some day, but for now I am hoping that this will do.
  (Ideally the BFD library's whole support infrastructure for relocs 
  should be rewritten, but that is a huge project).

Cheers
  Nick
Comment 4 franz.sirl 2020-08-20 07:23:48 UTC
Hi Nick,

yes, I can confirm that your patch brings back the performance to the original level.

Thanks,
Franz
Comment 5 Sourceware Commits 2020-08-20 09:20:53 UTC
The master branch has been updated by Nick Clifton <nickc@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=44466e45c51b1c68ed2f95f86bcdb1f18f08b380

commit 44466e45c51b1c68ed2f95f86bcdb1f18f08b380
Author: Nick Clifton <nickc@redhat.com>
Date:   Thu Aug 20 10:19:47 2020 +0100

    Apply a workaround to mitigate a quadratic performance hit in the linker when writing out secondary reloc sections.
    
            PR 26406
            * elf-bfd.h (struct bfd_elf_section_data): Add
            has_secondary_relocs field.
            * elf.c (_bfd_elf_copy_special_section_fields): Set the
            has_secondary_relocs field for sections which have associated
            secondary relocs.
            * elfcode.h (elf_write_relocs): Only call write_secondary_relocs
            on sections which have associated secondary relocs.
Comment 6 Nick Clifton 2020-08-20 09:21:34 UTC
Patch applied.
Comment 7 Romain Geissler 2020-08-31 08:54:55 UTC
Hi Nick,

Now that it has been in master for roughly 10 days, shall this be considered for a backport in the release branch 2.35 ?

Cheers,
Romain
Comment 8 Sourceware Commits 2020-09-03 15:01:54 UTC
The binutils-2_35-branch branch has been updated by Nick Clifton <nickc@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=790244d25897928fd5c950ee4e4ef84e3e9fd6ed

commit 790244d25897928fd5c950ee4e4ef84e3e9fd6ed
Author: Nick Clifton <nickc@redhat.com>
Date:   Thu Sep 3 16:00:48 2020 +0100

    Partially fix a quadratic slowdown when processing secondary relocations for inputs with lots of sections.
    
            PR 26406
            * elf-bfd.h (struct bfd_elf_section_data): Add
            has_secondary_relocs field.
            * elf.c (_bfd_elf_copy_special_section_fields): Set the
            has_secondary_relocs field for sections which have associated
            secondary relocs.
            * elfcode.h (elf_write_relocs): Only call write_secondary_relocs
            on sections which have associated secondary relocs.
Comment 9 Nick Clifton 2020-09-03 15:02:23 UTC
(In reply to Romain Geissler from comment #7)
> Now that it has been in master for roughly 10 days, shall this be considered
> for a backport in the release branch 2.35 ?

Done.