SHF_LINK_ORDER's original semantics make upgrade difficult

[Fangrui Song]

On 2020-07-16, Ali Bahrami wrote:
>On 7/16/20 1:27 AM, James Henderson wrote:
>>I've run into this same problem whilst doing some prototyping work on fragmenting DWARF sections into smaller pieces. These pieces are a mixture of common pieces and function/variable-related pieces, with the latter being associated with the corresponding text/data sections via SHF_LINK_ORDER. However, in some cases, the section is laid out such that there are multiple common pieces intermixed with the function/variable-specific pieces (e.g. it might look like "common, function, function, common, function, variable, common"). I actually don't want this section to be ordered at all, since the original ordering should be preserved.I've been using a hacked LLD for this, which removes the ordering for SHF_LINK_ORDER completely, so that the sections remain in the "natural" order, whilst the associated-with semantics for --gc-sections are still preserved.
>>
>>Whilst doing this, I couldn't help but feel that the associated-with and ordering semantics are somewhat orthoganol. Clearly if a section is ordered, it needs the associated-with semantics, but it seems like section association is a different thing. There are multiple sections now where the ordering is irrelevant, but which still wants association in some way. Examples include LLVM's stack sizes section and debug data.The original SHF_LINK_ORDER extension discussion actually started out with discussing a SHF_ASSOCIATED flag. Maybe we should revisit that idea in some form? Thus, SHF_ASSOCIATED implies the section should be discarded when it's linked section is discarded, and SHF_LINK_ORDER requires ordering. I think both for backwards-compatibility's sake and to avoid redundancy, we could say SHF_LINK_ORDER implies SHF_ASSOCIATED.
>>
>
>
>
>   I guess I don't understand why SHT_GROUP sections aren't the general
>solution to this scenario, and any others where sections need to be
>associated.
>
>    https://docs.oracle.com/cd/E37838_01/html/E36783/chapter7-26.html
>
>In your scenario, I think you'd set the flags field to 0, rather
>than GRP_COMDAT, and then you'd have an associated set of sections
>that are kept, or discarded, as a unit. And, GROUP conveniently go
>away as part of a "final" link that produces an executable or shared
>object, as you propose above.

If I understand correctly, linkers support GRP_COMDAT but not other values. So
you are right that we can make use of other values if we want to do some fancy
things.

In practice, I guess one reason people want to mess with a section flag is that
adding a .group has non-trivial cost (sizeof(Elf64_Shdr)=64) if an object file
may have many of them (especially when -ffunction-sections is enabled).

For example, I added -fpatchable-function-entry=N[,M] in clang for the Linux
kernel. Every .text section has an accompanying __patchable_function_entries
with SHF_LINK_ORDER set. If I want to express this with a section group, I need
to add a .group (SHT_GROUP) for each .text, the cost will go up from double to
triple.

>>On 7/16/20 1:27 AM, James Henderson wrote:
>> Whilst doing this, I couldn't help but feel that the associated-with and ordering semantics are somewhat orthoganol. 

Yeah, I am a big fan of non-orthogonality.. However, a gABI section flag seems a
no-go now.

>If the GNU linkers were modified to allow both ordered and
>non-ordered sections, then I think you could use GROUP for
>associating, and SHF_LINK_ORDER for actual ordering, where
>that is desired. We probably don't need a third mechanism,
>do we?

This solution is acceptable. I will do this for LLD
(https://reviews.llvm.org/D72904 ). We need an assembler syntax for
(SHF_LINK_ORDER & sh_link=0). 

I created  https://sourceware.org/bugzilla/show_bug.cgi?id=26253 for a GNU as feature request.
Peter Collingbourne proposed

   .section .meta,"ao",@progbits,0

I will expect that GNU as and LLVM integrated assembler match
(probably https://reviews.llvm.org/D72899#2157020 )