SHF_LINK_ORDER's original semantics make upgrade difficult

[James Henderson]

On Fri, 17 Jul 2020 at 00:32, Fangrui Song <maskray@google.com> wrote:

> On 2020-07-16, Ali Bahrami wrote:
> >On 7/16/20 1:27 AM, James Henderson wrote:
> >>I've run into this same problem whilst doing some prototyping work on
> fragmenting DWARF sections into smaller pieces. These pieces are a mixture
> of common pieces and function/variable-related pieces, with the latter
> being associated with the corresponding text/data sections via
> SHF_LINK_ORDER. However, in some cases, the section is laid out such that
> there are multiple common pieces intermixed with the
> function/variable-specific pieces (e.g. it might look like "common,
> function, function, common, function, variable, common"). I actually don't
> want this section to be ordered at all, since the original ordering should
> be preserved.I've been using a hacked LLD for this, which removes the
> ordering for SHF_LINK_ORDER completely, so that the sections remain in the
> "natural" order, whilst the associated-with semantics for --gc-sections are
> still preserved.
> >>
> >>Whilst doing this, I couldn't help but feel that the associated-with and
> ordering semantics are somewhat orthoganol. Clearly if a section is
> ordered, it needs the associated-with semantics, but it seems like section
> association is a different thing. There are multiple sections now where the
> ordering is irrelevant, but which still wants association in some way.
> Examples include LLVM's stack sizes section and debug data.The original
> SHF_LINK_ORDER extension discussion actually started out with discussing a
> SHF_ASSOCIATED flag. Maybe we should revisit that idea in some form? Thus,
> SHF_ASSOCIATED implies the section should be discarded when it's linked
> section is discarded, and SHF_LINK_ORDER requires ordering. I think both
> for backwards-compatibility's sake and to avoid redundancy, we could say
> SHF_LINK_ORDER implies SHF_ASSOCIATED.
> >>
> >
> >
> >
> >   I guess I don't understand why SHT_GROUP sections aren't the general
> >solution to this scenario, and any others where sections need to be
> >associated.
> >
> >    https://docs.oracle.com/cd/E37838_01/html/E36783/chapter7-26.html
> >
> >In your scenario, I think you'd set the flags field to 0, rather
> >than GRP_COMDAT, and then you'd have an associated set of sections
> >that are kept, or discarded, as a unit. And, GROUP conveniently go
> >away as part of a "final" link that produces an executable or shared
> >object, as you propose above.
>
> If I understand correctly, linkers support GRP_COMDAT but not other
> values. So
> you are right that we can make use of other values if we want to do some
> fancy
> things.
>
> In practice, I guess one reason people want to mess with a section flag is
> that
> adding a .group has non-trivial cost (sizeof(Elf64_Shdr)=64) if an object
> file
> may have many of them (especially when -ffunction-sections is enabled).
>
> For example, I added -fpatchable-function-entry=N[,M] in clang for the
> Linux
> kernel. Every .text section has an accompanying
> __patchable_function_entries
> with SHF_LINK_ORDER set. If I want to express this with a section group, I
> need
> to add a .group (SHT_GROUP) for each .text, the cost will go up from
> double to
> triple.
>
Fangrui more or less covered it, but to add, I have in the past
experimented with non-comdat groups, but since they weren't supported in
LLVM (no assembler syntax, no LLD support), it was a non-trivial amount of
work to get them to work nicely at the time. Plus, as mentioned, they are a
significantly higher file space (and likely link-time) overhead than just
using the sh_link value. Given that one of the main concerns with the debug
data fragmentation concept was file I/O overhead costs, due to there being
potentially many more section headers, adding another set per
function/variable (due to -ffunction-sections/-fdata-sections which is
where this approach would be used) makes the matter even worse.

Of course using the SHF_LINK_ORDER/a proposed SHF_ASSOCIATED flag does not
solve the question of what happens if sh_link is used for something else,
but that hasn't been relevant in my situation so far, since the sections
have all been SHT_PROGBITS.


>
> >>On 7/16/20 1:27 AM, James Henderson wrote:
> >> Whilst doing this, I couldn't help but feel that the associated-with
> and ordering semantics are somewhat orthoganol.
>
> Yeah, I am a big fan of non-orthogonality.. However, a gABI section flag
> seems a
> no-go now.
>
> >If the GNU linkers were modified to allow both ordered and
> >non-ordered sections, then I think you could use GROUP for
> >associating, and SHF_LINK_ORDER for actual ordering, where
> >that is desired. We probably don't need a third mechanism,
> >do we?
>
> This solution is acceptable. I will do this for LLD
> (https://reviews.llvm.org/D72904 ). We need an assembler syntax for
> (SHF_LINK_ORDER & sh_link=0).
>
> I created  https://sourceware.org/bugzilla/show_bug.cgi?id=26253 for a
> GNU as feature request.
> Peter Collingbourne proposed
>
>    .section .meta,"ao",@progbits,0
>
> I will expect that GNU as and LLVM integrated assembler match
> (probably https://reviews.llvm.org/D72899#2157020 )
>