Suppress the fetch of an archive member via --defsym (glibc/elf/librtld.map.o)

[Fangrui Song]

On 2020-04-11, Florian Weimer wrote:
>* Michael Matz:
>
>> On Mon, 16 Mar 2020, Fangrui Song via Binutils wrote:
>>> I am concerned that the --defsym's order dependence with archive files is not
>>> so obvious, given -u's behavior:
>>>
>>> # -u inserts an undefined which fetches b.a(b.o)
>>> ld.bfd -u foo b.a       # b.a(b.o) is fetched. free is present
>>> # This can't be order dependent because b.a (not in a group) should have been
>>> dropped when we saw -u
>>> ld.bfd b.a -u foo       # b.a(b.o) is fetched. free is present
>>>
>>> Some observations:
>>>
>>>
>>> # GNU ld --defsym interacts with an archive
>>> ld.bfd a.o b.a --defsym foo=0  # b.a(b.o) is fetched. free is present
>>> ld.bfd --defsym foo=0 a.o b.a  # b.a(b.o) is not fetched. free is absent
>>
>> I consider this the correct linker behaviour.  In that sense I would
>> consider the inconsistency with -u and -T to be the problems of those, not
>> a problem of --defsym (but not necessarily problems we can fix anymore, as
>> people might rely on that).  (i.e. I think the second -u command above
>> should result in an 'undefined foo' output/error, not in b.a(b.o) being
>> fetched).
>
>I agree, the ld.bfd behavior seems the most reasonable one to me.

ld accepts two types of command line arguments: input files / options.

Examples of order dependency between two input files:

* an earlier archive member can override a subsequent archive member
* a shared object can suppress the fetch of a subsequent archive member
* a fetched archive member (turns to an object file) overrides a shared object

Examples of order dependency between an input file and an option:

* --as-needed --no-as-needed
* --whole-archive --no-whole-archive
* --start-lib --end-lib
* -Bdynamic -Bstatic -static (I am saddened by https://sourceware.org/pipermail/binutils/2020-March/000110.html)
* --start-group --end-group

I acknowledge all the above. However, a large set of options have no
order dependency issues.

* --allow-shlib-undefined
* --dynamic-list
* --version-script
* --apply-dynamic-relocs
* --omagic
* ...
* -u
* --export-dynamic-symbol
* --exclude-libs

I am glad the options are designed this way, otherwise in people's build
systems they could notice many more hard-to-diagnose problems.

As my very first message said, if GNU ld had --start-lib for a long time
(feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=24600), this shadowing
malloc.o definition issue can be fixed very elegantly.

Now, we don't have --start-lib for GNU ld, but we can probably use GNU
ar thin archives as an alternative. The Linux kernel makes use of a
similar trick.


------- Random out of topic comments

In FreeBSD ports, 32k pieces of software just work, 130+ packages are
marked as LLD_UNSAFE, but many should be safe (need developers' testing)
as of lld 9. It is very likely none of the packages makes use of the
--defsym feature.

Most configurations of x86/x86-64/aarch64/ppc64le/ppc64le ports of the
Linux kernel link fine with LLD.

Making glibc compilable/linkable with clang/LLD is not at all my work
requirement.  It is simply I want to earn myself some contributions + I
don't want glibc to be tied to very specific toolchain. I started with
LLD because it is the relatively easy task. Clang compilability is more
troublesome. If I feel sufficient enmity, I will surely stop.

I tried stripping down LLD yesterday, just wanted to know how much code
is sufficient as the core https://github.com/MaskRay/picolld WoW, 18000+
LOC is capable to keep x86/ARM/AArch64/PPC ports and most functionality.
The GNU ld core probably really needs a refinement.

I have been thinking how to detect discrepancy between lld's and
traditional linkers' resolution rules. (The FreeBSD ports stats tell us
the "order dependency" issues are very few.) https://reviews.llvm.org/D77512

I am saddened that for some bug reports/enhancement proposals, the
initial reaction of some people in the community is usually "this is not
a bug." "this is an LLD bug" "GCC does this, so clang should do this as well."

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94391 is an awesome recent
showcase. It was then followed by an unfounded "NO LLD is not
implemented the ABI as PIE COPYRELOC is required by ABI these days".
The status was updated back and forth between "invalid", "wontfix" and
"worksforme" until a maintainer realized GNU ld had indeed one bug
(R_X86_64_[REX_]GOTPCRELX cannot be relaxed for SHN_ABS) and one
enhancement (PC relative relocations to a non-preemptible symbol should
be rejected). No one is probably willing to admit that LLD is good, though.