Dennis Ritchie's grace

[Fangrui Song]

On Thu, Jan 2, 2025 at 2:35 PM The Cuthour <cuthour@gmail.com> wrote:
>
> Previous thread
> https://sourceware.org/pipermail/binutils/2023-January/125568.html
>
>
> Do you think we have the ability to only recompile and relink modified
> files? Thanks to Dennis Ritchie's grace, that is universally true, but
> only in the context of structured programming. For example, in object-
> oriented programming, if you make changes to a file, you may need to
> recompile other dependent files that were not modified. In structured
> programming, everything is fine.
>
> The binary tools were written almost single-handedly by Dennis Ritchie,
> simultaneously with the C language, UNIX, and a.out format. That is the
> Dennis Ritchie's grace. That grace extended to structured programming
> in general, allowing languages like Pascal to benefit from it. However,
> since the advent of object-oriented programming, no one seems to be
> taking care of such conveniences anymore.
>
> Somebody not interested in?

Please could you write down your proposal in detail?

The wild linker (written in Rust) is exploring incremental linking
with a nice write up
https://davidlattimore.github.io/posts/2024/11/19/designing-wilds-incremental-linking.html

There are significant challenges and incremental linking would likely
trade off many important features:

* archive semantics
(https://maskray.me/blog/2021-06-20-symbol-processing#archive-processing)
* --gc-sections (https://maskray.me/blog/2021-02-28-linker-garbage-collection)
* SHF_MERGE duplicate elimination. Dropping this feature could
increase the .debug_str size a lot.
* .eh_frame elimination
* COMDAT
* ...

In addition, it seems very difficult to have deterministic result
results, which are required by several modern build systems.

Rust's lack of COMDATs and archive semantics (unused by rlib) and less
requirement on DSO/archive interaction simplifies the linking process
compared to C/C++.
However, this might not translate to the wild world.

On the other hand, full linking is already quite fast.
On a x86 machine, linking a release build of clang takes 400ms while
linking a non-debug build of chromium takes ~2s.
Many real world projects are smaller in scale, minimizing the
potential gains from incremental linking.

Some aspects are worth investigating beside the raw link speed.
Debug information can significantly impact file size and linking
speed. Linker features like --gdb-index and --debug-names are slow.
Exploring on-demand debug information generation, similar to Apple's
lazy scheme (https://maskray.me/blog/2022-10-30-distribution-of-debug-information#:~:text=lazy),
could be beneficial for typical development cycles (edit-compile-link-test).