GNU Tools Cauldron SFrame talk notes

[Indu Bhagat]

On 10/10/25 7:06 AM, Steven Rostedt wrote:
> On Fri, 10 Oct 2025 01:09:00 -0700
> Indu Bhagat <indu.bhagat@oracle.com> wrote:
> 
>>
>> Hi Fangrui,
>>
>> Kernel relocatable files is not the sole reason of the design choice of
>> having an index.  Without an index, stacktracers will have a run-time
>> hit of parsing the section to create an index or resorting to linear
>> scan; neither of these is acceptable for performance reasons.
>>
>>> The fundamental design issue lies in making linker merging mandatory.
>>> For optimal portability, unwinders should support multiple-element
>>> structures within a .sframe section. When a linker builds an index for
>>> .sframe, it should be viewed as an optimization that relieves the
>>> unwinder from constructing its own index at runtime. This index
>>> construction should remain optional rather than required. While the
>>> SFRAME_F_FDE_SORTED flag can be cleared to permit unsorted FDEs,
>>> current unwinder implementations do not seem to support multiple
>>> elements in a single section.
>>>    
>>
>> Constructing index at runtime will hurt.  Kernel's stacktracing of
>> userspace stacks will be done on the return to user path, and cannot
>> take that sort of performance hit.
> 
> Question:
> 
> Each object being linked would have a sorted sframe table, correct?
> 

ATM, we dont set the SFRAME_F_FDE_SORTED for object files output by 
assembler. It is the linker that decides the placement, so there was 
little incentive to do sorting earlier in the build pipeline..

> It's just that if the linker doesn't know what an sframe is, it would just
> concatenate all the sections placing several sorted sections together but
> the whole is not sorted?
> 

There can be scenarios where the linker may place functions from across 
objects in certain order (e.g., using linker-scripts or 
-ffunction-sections with section ordering 
https://developers.redhat.com/articles/2024/06/13/practical-guide-linker-section-ordering#linker_section_ordering). 
  There can be other function placement related optimizations that the 
linker may want to do...

The individual FDEs in each of input SFrame sections then are not 
correctly sorted in those cases anymore.  A consumer, then has no choice 
but to go through all SFrame sections and sort afresh.

Its not just that, if linker decides to discard a function definition 
(One Definition Rule, comdat group deduplication, etc.), an abiding 
linker needs to remove the corresponding SFrame FDE.  To resolve this in 
an "ELF compliant way" will be to have one SFrame section per .text* 
section.  Just saying that there will not be one SFrame section per 
object anymore, this means the number of concatenated SFrame sections 
will be the same as the number of .text* sections in the input object 
files: this can be large for some applications.

>>
>> Its not just the kernel, building the index at run-time will hurt _all_
>> other profiling and debugging use-cases where fast stack traces are
>> desirable.
>>
> 
> Is it possible to have the sframe header denote this? That is, if the
> linker knows about sframes, it would sort the entire section and set a flag
> in the sframe header that it is sorted. If not, then the kernel and other
> profilers could implement a slow path. Where at first read, it would create
> a secondary table of each sorted section, and sort that. This table would
> likely need to be allocated per task. We could possibly attach it to the
> inode of the executable so that if there's more than one instance of it
> running, they would all use the same table.
> 
> Then for searching the FDE via the instruction pointer, it would first look
> at the created outer table (which would be in sorted format such that it
> still can do a binary search) and then do a binary search of the inner table.
> 

Transferring the creation of index to stack tracers will be quite an 
overhead, especially for large binaries.  Duplicating this logic across 
the stack tracers is deterrent enough IMO.

> Yes, this may be more work on the reader of the sframe sections, but if
> this will simplify the linker side, it could be worth it.
>