[Bug find-debuginfo/33992] New: find-debuginfo script: processing large .a archives is extremely slow despite -j

[arekm at maven dot pl]

https://sourceware.org/bugzilla/show_bug.cgi?id=33992

            Bug ID: 33992
           Summary: find-debuginfo script: processing large .a archives is
                    extremely slow despite -j
           Product: debugedit
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: find-debuginfo
          Assignee: unassigned at sourceware dot org
          Reporter: arekm at maven dot pl
                CC: debugedit at sourceware dot org
  Target Milestone: ---

debugedit 5.3

find-debuginfo script takes 17+ minutes to process libint2.a (985x .o, 279MB)
on a 28- core machine. The -j flag - no effect - parallelism seem to only apply
across top-level files, not within a single archive.

do_ar_file() serializes debugedit runs. The while-read loop in do_ar_file()
calls debugedit on each extracted .o one at a time. At ~1s per call, 985
members = ~16 minutes, everythin serialized.

n_jobs gets clamped to the number of primary files before do_ar_file() is ever
called (n_jobs=$n_files). Is set to only 2 when a package ships just one .so
and one .a.

Possible fix: split the loop into 3 phases - extract-all,
debugedit-all-in-parallel, rebuild-in-order, using xargs -P  (debugedit using
max -j allowed). The -l SOURCEFILE output per-member files to avoid concurrent
writes, then merged.

And there is also issue with ar rebuilding.

Reproduction - a large static library containing hundreds of .o files with
debug info - in my case it was libint2.

-- 
You are receiving this mail because:
You are on the CC list for the bug.