SFrame FDE function start addr relocation
Jens Remus
jremus@linux.ibm.com
Mon Mar 10 14:39:33 GMT 2025
On 10.03.2025 08:38, Jan Beulich wrote:
> On 08.03.2025 08:48, Indu Bhagat wrote:
>> I will appreciate inputs on this SFrame related linker issue. The patch
>> series
>> "[RFC 0/4] Fix relocatable links with SFrame section" was recently posted:
>> https://sourceware.org/pipermail/binutils/2025-March/139853.html
>>
>> TL;DR: To resolve an issue, there is need to update r_addend of the SFrame
>> RELAs. This section-specific handling is being added to generic linker
>> code.
>> Does this affect current/future workflows in ld.bfd ? Are there other
>> options?
>> --------------------------------------
>> SFrame FDE function start addr relocation
>>
>> * SFrame FDE Function Start Address
>>
>> In SFrame V2, SFrame function descriptor entry is a 32-bit signed integer
>> (sfde_func_start_address) which is meant to somehow denote the start PC
>> of the
>> function.
>>
>> (Scheme #A) According to the current specification, it intends to hold the
>> offset of the start PC of the function from the
>> _start_of_the_SFrame_section_.
>> This value can then be used by stacktracers to simply do:
>>
>> sframe_find_fre (pc - sframe_vaddr)
>>
>> Where sframe_find_fre () can simply compare the value in each SFrame FDE's
>> sfde_func_start_address field, when looking up SFrame stack trace data
>> corresponding to the program counter (pc).
>>
>> (Scheme #B) Why not store the offset of the start PC of the function
>> from the
>> relocation place (_start_of_SFrame_FDE_) ? If done this way, an SFrame
>> stack
>> tracer will have to bear the cost of the extra step of offseting with
>> r_offset
>> for each FDE lookup; Something I thought could be avoided if we chose Scheme
>> #A.
>>
>> * Current Implementation and the Issue
>>
>> In the current implementation, however, GAS is actually emitting a
>> PC-relative
>> RELA such that sfde_func_start_address is the offset of the start PC of the
>> function from the _start_of_the_SFrame_FDE_ (Scheme #B!). In
>> gas/gen-sframe.c,
>>
>> /* Start address of the function. */
>> exp.X_op = O_subtract;
>> exp.X_add_symbol = dw_fde_start_addrS; /* to location. */
>> exp.X_op_symbol = sframe_start; /* from location. */
Nit: IIUC this is Scheme #A (relative to .sframe). For Scheme #B
(relative to FDE) it would be:
exp.X_op_symbol = symbol_temp_new_now (); /* from location. */
>> exp.X_add_number = 0;
>> emit_expr (&exp, addr_size);
>>
>> So the emitted relocations look like so:
>>
>> $ readelf -r temp.o
>> ...
>> Relocation section '.rela.eh_frame' at offset 0xaa58 contains 2 entries:
>> Offset Info Type Sym. Value Sym. Name
>> + Addend
>> 000000000020 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
>> 000000000040 000200000002 R_X86_64_PC32 0000000000000000 .text + 7
>>
>> Relocation section '.rela.sframe' at offset 0xaa88 contains 2 entries:
>> Offset Info Type Sym. Value Sym. Name
>> + Addend
>> 00000000001c 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
>> 000000000030 000200000002 R_X86_64_PC32 0000000000000000 .text + 7
>
> In this context it would help if the corresponding source was also shown.
> Without that it's hard to see why this is incorrect, while ...
>
>> But then, ld.bfd is then fixing up the relocated value of
>> sfde_func_start_address by adding the r_offset (in
>> _bfd_elf_merge_section_sframe ()). So finally what ld.bfd currently
>> generates
>> (for E_DYN and E_EXEC) is aligning to Scheme #A. Thinko it was, but moving
>> forward, this needs to be fixed because this essentially means we are
>> imposing
>> custom semantics of an existing PC-rel RELA relocation for E_REL binaries.
>> https://lore.kernel.org/linux-arm-kernel/20250225230137.620606-1-wnliu@google.com/T/#m1751e4abf1186ab2ae9064478d453d48542fe063
>>
>> * The fix: Use Scheme #A consistently
>>
>> If we fix the thinko in GAS described above, we now build the expression
>> `.text + text_offset - .sframe'. This expression results in GAS
>> emitting a PC-rel relocation such as:
>>
>> Relocated value = .text + text_offset - .sframe
>>
>> (S) .text |
>> |
>> -----------> | function foo \
>> | | |
>> | | | .text + text_offset - .sframe
>> | | |
>> | | /
>> ---> .sframe | \
>> | | P (sframe_offset)
>> | FDE (start_func_addr) /
>>
>> Where:
>>
>> S = .text
>> A = text_offset + sframe_offset
>> P = (.sframe +) sframe_offset
>>
>> Note how the addend A carries the sframe_offset (because it is the
>> r_offset of
>> the RELA) only to be annulled by substracting sframe_offset again when
>> calculating the relocation with S + A - P. This is GAS's way to encode this
>> "offset between two points one of which is the section containing the
>> PC/relocated-location". While the calculation is correct, it leads to
>> complications in the linker when linking to generate relocatable objects, as
>> explained later below.
>>
>> So with the fixed GAS, we will see the following:
>>
>> $ readelf -r temp.o
>> ...
>> Relocation section '.rela.eh_frame' at offset 0xaa58 contains 2 entries:
>> Offset Info Type Sym. Value Sym. Name
>> + Addend
>> 000000000020 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
>> 000000000040 000200000002 R_X86_64_PC32 0000000000000000 .text + 7
>>
>> Relocation section '.rela.sframe' at offset 0xaa88 contains 2 entries:
>> Offset Info Type Sym. Value Sym. Name
>> + Addend
>> 00000000001c 000200000002 R_X86_64_PC32 0000000000000000 .text + 1c
>> 000000000030 000200000002 R_X86_64_PC32 0000000000000000 .text + 37
>
> ... this would be correct. Furthermore ...
>
>> IMP: Note how the r_addend of the reloc in .rela.sframe is updated (and
>> different compared to the r_addends shown for .rela.eh_frame) by
>> including the
>> r_offset in it. The actual "text_offset" is now merged with the "r_offset".
>> This brings in link-time considerations as we will see later.
>>
>> * Linking Considerations with the fix
>>
>> To set the context for the linking process, note how information is
>> organized
>> in SFrame sections:
>>
>> --------------------- ---------------------
>> | SFrame Header | | SFrame Header |
>> --------------------- ---------------------
>> | SFrame FDE 1a | (Linking) | SFrame FDE 1a |
>> | SFrame FDE 1b | | SFrame FDE 1b |
>> | ... | |------>| ... |
>> | SFrame FDE 1n | | | SFrame FDE 1n |
>> --------------------- | | SFrame FDE 2a |
>> | ... | ------ | SFrame FDE 2b |
>> |SFrame FREs (Sec 1)| | | ... |
>> |(Frame Row Entries)| | | SFrame FDE 2n |
>> | ... | | |-------------------|
>> | ... | | | ... |
>> --------------------- | |SFrame FREs (Sec 1)|
>> | |SFrame FREs (Sec 2)|
>> | |(Frame Row Entries)|
>> | | ... |
>> --------------------- | ---------------------
>> | SFrame Header | |
>> --------------------- |
>> | SFrame FDE 2a | |
>> | SFrame FDE 2b | |
>> | ... | ------
>> | SFrame FDE 2n |
>> ---------------------
>> | ... |
>> |SFrame FREs (Sec 2)|
>> |(Frame Row Entries)|
>> | ... |
>> | ... |
>> ---------------------
>>
>> Figure 1 SFrame information layout and linking process
>>
>> Unlike EH_Frame, where a DWARF FDE (containing its associated DWARF
>> instructions) is relocated as a single unit, SFrame FDEs and FREs are
>> grouped
>> separately. A function's stack trace data requires one SFrame FDE and
>> multiple
>> SFrame FREs. This structure impacts how ld.bfd outputs RELAs for SFrame
>> sections during relocatable links.
>>
>> * Relocatable Links and r_addend Update
>>
>> With the fixed GAS following Scheme #A, a PC-rel relocation is generated to
>> (correctly) express the offset between the PC and the section containing the
>> relocated-location. This is achieved by adding r_offset (P) to A, and
>> when the
>> relocation is resolved (S + A - P), it effectively cancels out P,
>> yielding the
>> desired result.
>>
>> This, however, has implications for relocatable links: we need to manually
>> calculate the r_addend for the output RELAs. We now have to manually
>> "extract"
>> the "text_offset" from the input-reloc's A. We then need to add to it
>> the new
>> r_offset of the SFrame FDE in the output SFrame section to finally
>> obtain the A
>> for the output reloc.
>
> ... such custom handling shouldn't normally be required. Everything would
> better come out "natural".
IIUC a PC-relative relocation is expected to be relative to the location
of its associated field. In Scheme #A it is used (for the FDE function
start address - think of it as function start offset) as if it was
relative to the section its associated field resides in. This causes
the offset of the associated field from its section to be baked into the
addend.
Merging of SFrame sections gathers all FDEs from the input sections and
places them together in the output section. In any case this requires
to update the relocation r_offset, as the location of the field changes.
For Scheme #A this also causes the baked-in field offset from section in
the addend to change, making it necessary to update the addend.
@Indu:
Sorting of the FDEs (based on FDE function start address) is also
performed for relocatable links. IIUC it is effectively skipped, as
the (non-relocated) FDE function start address is zero for all FDEs,
so that fde_func() always compares equal and qsort() does nothing in
sframe_sort_funcdesc().
Wouldn't it make sense to explicitly skip sorting of FDEs for
relocatable links? I also wonder whether the FDEs in a relocatable
link are really always sorted. It appears to be true for single output
.text section, as input .text sections then get concatenated in the
order they appear as input (unless there is some way to change this via
linker script?). But if there are multiple .text sections (e.g. .text
and .text.hot) the output does not look right for relocatable links.
>> This additional logic is carved out as a separate patch "[RFC 3/4] ld: bfd:
>> sframe: fix incorrect r_offset in RELA entries" to aid discussion. As
>> you see
>> this is being done in generic linker code in elf_link_input_bfd ().
>>
>> * The r_addend Update Issue
>>
>> If such a "r_addend fixup" is risky or wrong, it seems we will need a
>> new type
>> of RELOC for SFrame sections ? This is the reason the series is marked
>> as RFC.
>> I am not completely clear on whether doing so has impact on current/future
>> workflows in ld.bfd.
>>
>> * Miscellaneous
>>
>> The update of irela->r_offset in elf_link_input_bfd() is necessary,
>> irrespective of scheme #A or #B, as explained in "[RFC 3/4] ld: bfd:
>> sframe: fix
>> incorrect r_offset in RELA entries".
>>
>> As depicted in Figure 1, the offset of an SFrame FDE in the output
>> section is
>> not simply "input_section->output_offset + r_offset". Therefore, the
>> following
>> explicit stub is required in elf_link_input_bfd():
>>
>> if (o->sec_info_type != SEC_INFO_TYPE_SFRAME)
>> irela->r_offset += o->output_offset;
>>
>> IOW, "[RFC 3/4] ld: bfd: sframe: fix incorrect r_offset in RELA entries" is
>> essential to resolve PR 32666 and enable relocatable links, regardless
>> of the
>> chosen scheme. I include it in the series as I need the relocatable
>> links to
>> work to later show the r_added related stubs.
Regards,
Jens
--
Jens Remus
Linux on Z Development (D3303)
+49-7031-16-1128 Office
jremus@de.ibm.com
IBM
IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Böblingen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/
More information about the Binutils
mailing list