Bug 27452 - ld: Support compressing arbitrary sections (generalized --compress-debug-sections=)
Summary: ld: Support compressing arbitrary sections (generalized --compress-debug-sect...
Status: UNCONFIRMED
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-02-21 23:30 UTC by Fangrui Song
Modified: 2024-03-12 00:33 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Fangrui Song 2021-02-21 23:30:51 UTC
ld --compress-debug-sections=zlib compresses the .debug_* output sections (usually non-SHF_ALLOC, but I've seen rustc SHF_ALLOC .debug_gdb_scripts - which could be a rustc bug).

There are needs compressing other sections, even SHF_ALLOC ones, e.g. __llvm_covstr for potential future clang coverage mapping formats (https://bugs.llvm.org//show_bug.cgi?id=49155). For SHF_ALLOC sections, there will be additional requirement on the runtime side as the runtime needs to figure out whether the section is compressed or not (perhaps just checking the Elf*_Chdr header (section header table is optional in executable/shared object files))

If we don't distinguish SHF_ALLOC and non-SHF_ALLOC sections, the new option can be '--compress-sections=.debug_*=zlib' . If we just need non-SHF_ALLOC sections, the new option can be '--compress-nonalloc-sections=.debug_*=zlib' . The exact requirements may need fleshing out.
Comment 1 Fangrui Song 2023-06-27 06:19:49 UTC
I think we should just allow SHF_ALLOC | SHF_COMPRESSED sections. Created https://groups.google.com/g/generic-abi/c/HUVhliUrTG0

The proposed option syntax is: --compress-sections='.debug_*=zlib' . This applies to both SHF_ALLOC and non-SHF_ALLOC sections.
Comment 2 Fangrui Song 2024-03-12 00:33:04 UTC
I find it very difficult to handle SHF_ALLOC sections due to data commands, range extension thunks, and other features requiring fixed-point iteration.

Compressing non-SHF_ALLOC sections alone may yield a significant value, e.g. compressing .symtab and .strtab .

On the lld side, I've created https://github.com/llvm/llvm-project/pull/84855 .  We are debating whether the option should be named 
--compress-sections <section-glob>=[none|zlib|zstd] or --compress-nonalloc-sections