[PATCH] dependency list for static libraries

[Fangrui Song]

Thanks to Nick for mentioning me :)

On 2020-09-23, Howard Chu wrote:
>Nick Clifton wrote:
>> Hi Howard,
>>
>>> Sorry for the noise, this attached is the same as previous but with fixes
>>> to whitespace / indentation.
>>
>> Thanks - the patch looks good now, although there are still a couple of minor
>> issues - and one major issue:
>>
>>   * The new test fails for the alpha-vms target.  This is not serious however
>>     as almost all of the ar tests fail for this target.  One day I will track
>>     down what is going wrong and either fix it, or arrange to skip these tests
>>     for alpha-vms.
>>
>>   * There are a couple of minor formatting issues.  Specifically: comments should
>>     end in a period followed by two spaces before the closing marker. /* Like this.  */
>>     Plus function calls should have a space between the function name and the
>>     opening parenthesis of the argument list.  like (this)
>
>I can send an updated patch after the major issue is addressed.
>
>> The major issue is that I would really like for this extension to be supported
>> by the LLVM community as well.  It would be a shame to add it to the binutils
>> only to have a different solution implemented there.  I might have misread the
>> emails but I believe that Fangrui still has some misgivings about this approach.
>> Is that correct ?  If so, then I would very much like to see them resolved
>> before we commit the patch.
>
>It still seems pretty obvious that handling dependencies once at the library
>level is more efficient than sticking a bunch of free-form notes in every single
>object file. And it should also be obvious that choice of library is a buildtime
>decision, not a decision made solely at the time the source code is written.
>(And for shared libraries, it's a runtime decision - even further removed from
>the time of code being written.)

 From my experience (I have investigated hundreds of link issues),
archive link errors are usually caused by the following two problems:

* There is a backward reference between two archives. This is related to a.out/ELF style linker behaviors
(https://sourceware.org/pipermail/binutils/2020-September/113194.html ).
   + A dependency exists but is incorrectly omitted. The linker actually captures a layering problem.
   + A dependency is intentionally omitted. If the linker allows backward references, this will not be a problem.

   I have recently thought a lot on the topic and written http://lld.llvm.org/ELF/warn_backrefs.html

* Some archives are incorrectly omitted: the executable uses A, but A's
   dependency B is not on the command line.  Now I hope Howard can clarify on this
   topic:) What do you want to achieve with the dependency recorded in the archive?
   I assume that you want the linker to smartly link B. This is indeed the #pragma
   comment(lib, ...) and .deplibs feature clang/LLD support.

   The proposal does not mention the ld part, which seems important. How
   does ld handle __.LIBDEP ?

   + The executable also references B and you don't want to write its dependency on B.
     This is the traditional --copy-dt-needed-entries behavior (that the
     designers of gold explicitly do not want to support). The linker traverses all
     dependencies of shared objects recursively. This is against the "include what
     you use" philosophy.  https://github.com/include-what-you-use/include-what-you-use/blob/master/docs/WhyIWYU.md
     It imposes a lot of extra work for the linker. It makes the build brittle - if
     A removes its dependency on B, you need to find and fix all the transitive users
     of A. I hope this proposal is not about the bullet point.
   + The executable does not references B. I am still unsure why you want to move the dependency information from the
     object file to the archive.
     - First, I want to mention that in some cases archives are not needed. For
       some use cases people propose thin archives.  For some thin archive use cases,
       --start-lib --end-lib surrounded object files are probably more useful.  gold
       pioneered the idea but GNU ld still does not support the feature
       https://sourceware.org/bugzilla/show_bug.cgi?id=24600
     - Moving the dependency to the archive can actually increases the work for
       the linker.  Say the two members of A are a0.o and a1.o. If a0.o depends on B
       but a0.o is not selected, then B does not need to be linked. Having the
       dependency in the archive will cause B to be linked.
     - How do you ensure the dependency information does not become stale? i.e.
       How do you maintain the dependency information when members are added to/removed from
       the archive? The information is more difficult to maintain than the index, which only contains
       the definitions. You may argue that the archive is finalized after it is created - then
       we go back to square one, --start-lib may be more suitable.

---

Last,

>  fprintf (s, _("  [L LIBDEPS]  - specify dependencies of this library\n"));

In llvm-ar, 'L' is an extension used with 'q' to add all members of an archive to the current archive.
It is similar to ADDLIB (https://sourceware.org/binutils/docs/binutils/ar-scripts.html )
Hope the two tools don't have conflicting operations.