Supporting > 2GiB text sections in SFrame/EH_Frame

[Indu Bhagat]

Hi,

One of the desirable features for SFrame stack trace format is to 
support applications with text > 2 GiB to make it future proof.

https://sourceware.org/binutils/wiki/sframe/sframev3todo#Support_applications_with_.text_.3E_2_GiB

https://lore.kernel.org/lkml/CAKwvOdmZYFhz0djG0_CgQ94BLaW8rUmdW+zaoZ0G+r76Esf2+Q@mail.gmail.com/

Where does the limit come from?

Each SFrame FDE needs to hold information about the start PC of the 
function it pertains to.  Currently, the field 'sfde_func_start_address' 
in SFrame FDE is encoded as a (signed) 32-bit offset to the start PC of 
the function from the field itself.  Keeping an offset helps with 
keeping the section free of dynamic relocations for adjusting these 
values as the final address of text segment may be unknown until load time.

Currently this offset is a signed 32-bit offset.  The signedness gives 
the flexibility of having .sframe ELF section before or after the .text* 
sections. But the size of 32-bit puts the limitation that .sframe 
together with the .text* sections must fit the 2 GiB range.

FWI, ATM, both the SFrame and EH_Frame formats in GNU Binutils (gas and 
ld) use signed 32-bit offsets to reference function start addresses. Let 
me call these offsets "addr_offset". This limits the size of the 
combined .text and .eh_frame/.sframe sections to 2GiB, as distances 
greater than this cause a "relocation truncated to fit" error.

Can SFrame be extended to relax this limitation without bloating up the 
format, and keeping its basic features in place that help with fast 
stack tracing?

Yes, we can add a new SFrame FDE representation, say, struct 
sframe_func_desc_entry_64bit (which uses int64_t sized addr_offset, i.e. 
int64_t sfde_func_start_address) and add a flag in the header when this 
FDE is in use.  BTW, in SFrame format, the SFrame FDE are fixed-sized 
objects with naturally-aligned members.

So what is the issue?

The issue is: what should be done when mixing sections with 32-bit 
addr_offset with 64-bit addr_offsets ?

Consider a link for a large application whose objects a.o and b.o use 
64-bit addr_offsets.  But an application will also bring in some runtime 
components.  E.g., :
    ld a.o b.o <XYZ>/lib64/crt1.o <XYZ>/lib64/crti.o
These crt*.o objects or any other runtime objects from the system may 
(likely) be 32-bit addr_offsets.  Now if the linker places the text 
sections from these runtime objects after the text of a.o/b.o, there may 
be a relocation overflow because the crt*.o use 32-bit addr_offsets and 
hence 32-bit relocations.

This means mixing 32-bit addr_offsets with 64-bit addr_offsets is not 
something the linker can easily handle, as it cannot "upgrade" the 
32-bit locations to 64-bit locations in section contents.

So lets say all objects are forced to use 64-bit addr_offsets for now. 
With that, lets say somehow we can resolve the matter for SFrame:
   - add a new --gsframe64 option to gas to use 64-bit addr_offsets for 
SFrame.  (Also add --gsframe32 to keep user interface flexible).
   - keep --gsframe64 option enabled the default (and not --gsframe32).
   - the linker when linking SFrame sections will only see 64-bit 
addr_offsets.  But after linking, it can determine if 32-bit offsets 
suffice and hence, can "relax" the addr_offsets for most applications to 
use 32-bit offsets; and adjust the size of the output section etc. 
Except for relocatable links, which will continue to use 64-bit 
addr_offsets.

(The above is a rough sketch of what can be considered a "POC" for now. 
My main concern is stated next.)  But assume , _somehow_ above can be 
worked out for SFrame, what should be the solution for 
.eh_frame/.eh_frame_hdr for supporting applications with text > 2 GiB. 
The specification of the EH_Frame format does allow 64-bit addr_offsets, 
IIUC.  The issue is how should the implementation in GNU ld/as be 
managed for .eh_frame to support applications needing text > 2 GiB. 
Without solving the issue for EH_Frame, links of such applications will 
continue to fail with or without SFrame.

Thanks
Indu