GNU Tools Cauldron SFrame talk notes

[Jose E. Marchesi]

>> On Oct 9, 2025, at 8:14, Indu Bhagat <indu.bhagat@oracle.com> wrote:
>> 
>> Hi,
>> 
>> A brief summary of two SFrame-related items discussed at GNU Tools Cauldron
>> 2025 (talk on "SFrame for effective userspace stack tracing"
>> https://conf.gnu-tools-cauldron.org/opo25/talk/ADVMH7/). More details on SFrame
>> stack trace format can be found on the wiki [1].  Thanks for your inputs during
>> the session and all the hallway chats.
>> 
>> * Handling in SFrame unaware linkers
>> 
>>  For the case of distro-wide enablement where the assembler generates an
>>  SFrame section, but the used linker does not support merging of SFrame
>>  sections, following link-time failures will be seen:
>> 
>>    error: relocation refers to a discarded section: XYZ
>> 
>>  The error is in accordance with the ELF specification (handling of unknown
>>  section type), and the eventual resolution is to support merging of SFrame
>>  sections.
>> 
>>  As per the ELF specification
>> (https://gabi.xinuos.com/elf/03-sheader.html#rules-for-linking-unrecognized-sections):
>>   - "Unrecognized sections that do not have the SHF_OS_NONCONFORMING attribute,
>>     are combined in a two-phase process."
>>   - "In the first phase, input sections that match in name, type and attribute
>>     flags should be concatenated into single sections."
>> 
>>  We discussed how this default behaviour proves to be a limitation to allow
>>  new section types (SFrame being one example), where
>>   - the new section type's layout is such that concatenated output is not
>>     meaningful/useful.
>>   - not all link-editors may want to deal with the new section type.
>> 
>>  It may be worthwhile to explore the possibility of adjusting the ELF
>>  specification to have SFrame sections be marked with a new flag, such that
>>  the default linker behavior of concatenating the sections can be skipped.
>> 
>>  The above issue is a blocker for SFrame enablement on some distros. Further,
>>  some distros reportedly have as many as 8 linkers.  If ELF can be adjusted, the
>>  currently SFrame-unaware linkers can discard the SFrame section.
>> 
>>  After the discussion at Cauldron, I think the following addition to ELF may
>>  work:
>> 
>>  SHF_OS_NONCONFORMING_DISCARD (new flag)
>>  This section requires special OS-specific processing (beyond the standard
>>  linking rules) to avoid incorrect behavior. If this section has either an
>>  sh_type value or contains sh_flags bits in the OS-specific ranges for those
>>  fields, and a link editor processing this section does not recognize those
>>  values, then the link editor should discard the section without an error.
>> 
>>  WRT relocations, I think the following may work (does this need to be
>>  specified?):
>>  Any section linked via sh_link must also be discarded.  Sections with flag
>>  SHF_OS_NONCONFORMING_DISCARD cannot have any relocations referring to these
>>  sections from outside of the section.
>> 
>>  WDYT ?
>> 
>> * Linker-friendly SFrame section layout
>> 
>>  We also discussed the question of whether it is possible to make SFrame
>>  handling linker-friendly.
>> 
>>  One of the possible options was to split SFrame section so that link-editors
>>  can assemble final section without needing to understand the contents of the
>>  section.
>> 
>>  Separate SFrame data into three sections
>>  - SFrame header (.sframe_header)
>>    flags = SHF_ALLOC
>>  - SFrame function index (FDEs sorted on start PC) (say, .sframe_idx),
>>    flags = SHF_LINK_ORDER | SHF_ALLOC
>>  - SFrame FRE data (concatenable, say .sframe_info)
>>    flags = SHF_ALLOC
>> 
>>  Use linker script to place the header first, followed by .sframe_idx, and
>>  finally .sframe_info.  For linkers which work without linker script, they
>>  usually assume sections are placed in alphabetical order.  But since we need
>>  the linker script to emit some vital section (header) data like number of FDEs,
>>  using linker scripts has a major drawback that SFrame will simply be
>>  unsupported on some linkers.  Its also not clear how/where to emit one header
>>  with all the necessary bits like number of FDEs and other offsets. Getting rid
>>  of the header or duplicating the header SFrame FRE data is a no-go:
>>   - SFrame header holds vital information like version, flags and other
>>     provisions necessary for format evolution
>>   - It is wasteful to duplicate it for each set of function FREs (in
>>     .sframe_info).  It will bloat up the format considerably.
>> 
>>  So, it seems the general advise was to not rely on linker script mechanism.
>>  Above all, making the layout linker-friendly with this approach adversely
>>  affects the efficacy of the format.
>
> In light of recent discussions, I was re-reading the thread and thought this
> might be worth revisiting.
>
> The advantage of using existing linker constructs is that you have a clear
> contract between SFrame and the linker. The linker is not in charge of
> providing compat; if compat works through that linker construct, then you have
> forward and backward compat.

That is clearly the best possible outcome.

I think we all agree in that it would be great if we could come with a
concatenable format for SFrame, _provided_ it stays simple, compact and
expressive enough for the requirements of the kernel userland unwinder
and similar clients.

> Merging the header is indeed harder. I imagine that one would take the maximum
> version across all object files and union of the flags, but AFAIK there are no
> ELF constructs for that (closest is IMAGE_COMDAT_SELECT_LARGEST in PE). So it
> seems reasonable to have custom merging for the header only. I strongly suggest
> specifying the linker contract (i.e. merging rules) in the spec, e.g. how each
> field should be merged, to avoid the slippery slope of “linkers DTRT” (which is
> not possible if you don’t specify DTRT before upgrading the spec).

I think it would be possible to have a concatenable header if it would
be defined in terms of a variable-length sequence of tags.  Something
like:

      .sframe_hdr
   +------+---...--+
   | tag  |  value |
   +------+--...---+
   | tag  |  value |
   +------+--...---+
         ...

With tags like:

  version (length 1), flags (length 4), abi_arch (length 1),
  fixed_fp_offset (length 1)

i.e. the stuff we currently have in struct sframe_header.  This way the
concatenation of several headers (sets of tags) would be a valid header
(the union set of tags).  It is a flexible layout when it comes to add
new tags in future versions of the format.

This would obviously complicate the client side to some extent... the
clients would need to use little loops in order to fetch tag values from
headers, checking for integrity on the way: all version tags in the
header shall be the same, the different flags shall be coherent, etc.
It would be important to determine whether this extra complexity would
be assumible by the existing clients wanting to use the format.  The
presence of variable-length structures in the format may make it
difficult (or even impossible) to process SFrame with verifiable BPF for
example.  Also, along with a pointer to the header you would need its
size.

But lets assume we indeed have concatenable headers like sketched above
or by some other mean.  Then we would need to split the sframe
generation in the assembler into two sections:

   .sframe_hdr   -> concatenable header
   .sframe       -> set of FDEs followed by a set of FREs

Like .eh_frame, the single .sframe section contains data for all text
sections in the object.  Then AFAIK at least two issues would remain:

1. An explicit index generated at link-time will be necessary.  In the
   current SFrame design, the index is implicit since the FDEs are well
   demarcated (by the header) and ordered, so clients can just binary
   search there.

   This will still require linker awareness.

2. Input text sections covered by the unique .sframe may be discarded,
   leading to an error.

   Roland just suggested a possible solution for this based on section
   groups:
   https://groups.google.com/g/generic-abi/c/3ZMVJDF79g8/m/udrN4dc3AAAJ

   Alternatively we could have the Sframe to somehow refer to the
   covered sections and make the linker using that info.  This would
   require linker awareness.

Any idea on how to handle these issues, while keeping the format
suitable for its intended usage?

> Tatsuyuki
>
>> 
>> [1] SFrame wiki:https://sourceware.org/binutils/wiki/sframe
>> 
>>