Supporting > 2GiB text sections in SFrame/EH_Frame

[Indu Bhagat]

Hi Sterling,

Thanks for your suggestions,

On 8/28/25 8:50 AM, Sterling Augustine wrote:
>> One of the desirable features for SFrame stack trace format is to
>> support applications with text > 2 GiB to make it future proof.
> 
> Strongly agreed that this would be a good thing to do.
> 
>> So lets say all objects are forced to use 64-bit addr_offsets for now.
>> With that, lets say somehow we can resolve the matter for SFrame:
>>     - add a new --gsframe64 option to gas to use 64-bit addr_offsets for
>> SFrame.  (Also add --gsframe32 to keep user interface flexible).
>>     - keep --gsframe64 option enabled the default (and not --gsframe32).
>>     - the linker when linking SFrame sections will only see 64-bit
>> addr_offsets.
> 
> It would be really good to avoid an extra-four bytes per FDE, even in the 64-bit
> case--if you need 64-bit offsets, then you likely have a lot of functions, and
> an extra four-bytes per function ends up adding a lot of size.
> 
> I think this can be fixed by reworking the sframe64_func_desc_entry
> into a sixteen-bytes structure.
> 
> 1. Move function size info into the FREs, eliminating the sfde_func_size field.
> 
> FREs already contain 99% of this size info in their offsets. There are a couple
> of ways we could add the total size in the FREs.
> 
> A. A simple one would be to have the first X bytes at
> sfde_func_start_fre_off be the total size of the function, instead of
> an FRE.  If the fretype is ADDRX, then the first X bytes are the total size.
> This makes it easy to find the total size, at the expense of the FREs being
> slightly-non-uniform.
> 

To me, this is simpler than B. But the catch here is that FRE 
sub-section, by design, favors compaction.  This means that the FRE data 
may be unaligned (this includes sfre_start_address and the stack 
offsets).  With the function size now going to the FRE sub-section, this 
may be now an additional unaligned access...

> B. We can add an FRE type that presents the size in
> sfre_start_address. Currently fre_offset_size has an unused fourth
> value that could mark this case. These FREs would also have an
> offset-count of zero so no extra data follows. These FREs should probably
> go first or last in the list. First would be nice because it makes finding
> the size easy. Last would be nice because parsing needs no special cases with
> an sfre_start_address out of place.
> 

Function size needs to be easily accessible for stack tracers to do 
binary search effectively.  Because there can be holes between PC ranges 
per function, it (function size) is a vital piece of info needed to do 
the work of "find the FDE for a given PC".  So I would say, if we go 
this route, this should be the first FRE.

BTW, tangential to the current discussion at hand, but somewhat related 
to the idea of "offset count of zero", we have been considering to use 
an offset count of zero to indicate outermost frame in SFrame.
https://sourceware.org/pipermail/binutils/2025-July/143040.html

> This does make things somewhat harder for consumers to discover when
> there is no sframe data for a function, but not substantially. They
> find the last FDE with an address lower than the PC, and then find the
> size, wherever that is encoded.
> 

Overall though, I would attempt to keep the separation between what FDEs 
are meant for (function level metadata) vs what FREs are meant for (Per 
PC range stack trace metadata).

So my suggestion would be not go with B at all.

> 2. Change sfde_func_num_fres to a uint16_t.
> 
> I expect that functions with more than 65,000 fres are extremely rare
> (and linear scans through that many fres is going to be slow
> regardless). But in the event this field overflows, the fix is simple
> and can be done at assembly time:
> 
> Terminate the current FDE and start a new, duplicate FDE at the current
> FRE offset.
> 

ACK, uint16_t for num_fres is doable.

> 3. Elminate the padding2 uint16_t.
> 
> This is completely harmless.
> 

Yes, this was added to keep alignment for members of FDE.  BTW, also 
serving as bytes that can be used as format evolves.  Can be removed if 
we go this route.

> 
> The new structure would look like this:
> 
> typedef struct sframe64_func_desc_entry
> {
>    int64_t sfde_func_start_address;
>    uint32_t sfde_func_start_fre_off;
>    uint16_t sfde_func_num_fres;
>    uint8_t sfde_func_info;
>    uint8_t sfde_func_rep_size;
> } ATTRIBUTE_PACKED sframe64_func_desc_entry;
> 
>>   But after linking, it can determine if 32-bit offsets suffice and
>> hence, can "relax" the addr_offsets for most applications to use
>> 32-bit offsets; and adjust the size of the output section etc.
>> Except for relocatable links, which will continue to use 64-bit
>> addr_offsets.
> 
> This idea makes sizeof(sframe64_func_desc_entry) == 16, which is
> four-bytes smaller than the current 32-bit version.
> 
> In the common case (small function, small cfa offsets), the size data
> is only 1 byte with scheme A, and two bytes with scheme B. In the
> worst case (large function, large cfa offsets), the size data is four
> bytes with scheme A, and five bytes with scheme B.
> 
> There would be no need to relax 64-bit fdes into 32-bit fdes: 64-bit
> FDEs are more efficient, and therefore to be preferred.
> 
> All of this hinges on the acceptability of moving the size field. But
> I think the tradeoff to avoid 8-bytes per FDE is a good one.
> 
> I suspect the implementation complexity this scheme adds is lower than
> the implementation complexity the relaxation scheme adds.
> 

Not relaxing 64-bit FDE to 32-bit FDE will be less complexity, no doubt.

I will evaluate the above suggestion with a POC soon.

BTW, for a subset of binaries generated for GNU Binutils package, my 
back-of-the-envelope calculations suggest a 6-7% increase in SFrame 
section size if using 64-bit FDEs over 32-bit FDEs.

Thanks
Indu

>> Without solving the issue for EH_Frame, links of such applications will
>> continue to fail with or without SFrame.
> 
> Eh_Frame also seems like a problem worth solving, but for our uses
> cases at least, sframe is more urgent.
> 
> Thanks for considering,
> 
> Sterling