SFrame FDE function start addr relocation
Indu Bhagat
indu.bhagat@oracle.com
Mon Mar 10 17:03:53 GMT 2025
On 3/10/25 7:39 AM, Jens Remus wrote:
> On 10.03.2025 08:38, Jan Beulich wrote:
>> On 08.03.2025 08:48, Indu Bhagat wrote:
>
>>> I will appreciate inputs on this SFrame related linker issue. The patch
>>> series
>>> "[RFC 0/4] Fix relocatable links with SFrame section" was recently
>>> posted:
>>> https://sourceware.org/pipermail/binutils/2025-March/139853.html
>>>
>>> TL;DR: To resolve an issue, there is need to update r_addend of the
>>> SFrame
>>> RELAs. This section-specific handling is being added to generic linker
>>> code.
>>> Does this affect current/future workflows in ld.bfd ? Are there other
>>> options?
>
>>> --------------------------------------
>>> SFrame FDE function start addr relocation
>>>
>>> * SFrame FDE Function Start Address
>>>
>>> In SFrame V2, SFrame function descriptor entry is a 32-bit signed
>>> integer
>>> (sfde_func_start_address) which is meant to somehow denote the start PC
>>> of the
>>> function.
>>>
>>> (Scheme #A) According to the current specification, it intends to
>>> hold the
>>> offset of the start PC of the function from the
>>> _start_of_the_SFrame_section_.
>>> This value can then be used by stacktracers to simply do:
>>>
>>> sframe_find_fre (pc - sframe_vaddr)
>>>
>>> Where sframe_find_fre () can simply compare the value in each SFrame
>>> FDE's
>>> sfde_func_start_address field, when looking up SFrame stack trace data
>>> corresponding to the program counter (pc).
>>>
>>> (Scheme #B) Why not store the offset of the start PC of the function
>>> from the
>>> relocation place (_start_of_SFrame_FDE_) ? If done this way, an SFrame
>>> stack
>>> tracer will have to bear the cost of the extra step of offseting with
>>> r_offset
>>> for each FDE lookup; Something I thought could be avoided if we chose
>>> Scheme
>>> #A.
>>>
>>> * Current Implementation and the Issue
>>>
>>> In the current implementation, however, GAS is actually emitting a
>>> PC-relative
>>> RELA such that sfde_func_start_address is the offset of the start PC
>>> of the
>>> function from the _start_of_the_SFrame_FDE_ (Scheme #B!). In
>>> gas/gen-sframe.c,
>>>
>>> /* Start address of the function. */
>>> exp.X_op = O_subtract;
>>> exp.X_add_symbol = dw_fde_start_addrS; /* to location. */
>>> exp.X_op_symbol = sframe_start; /* from location. */
>
> Nit: IIUC this is Scheme #A (relative to .sframe). For Scheme #B
> (relative to FDE) it would be:
>
> exp.X_op_symbol = symbol_temp_new_now (); /* from location. */
>
Thanks for the critical correction.
>>> exp.X_add_number = 0;
>>> emit_expr (&exp, addr_size);
>>>
>>> So the emitted relocations look like so:
>>>
>>> $ readelf -r temp.o
>>> ...
>>> Relocation section '.rela.eh_frame' at offset 0xaa58 contains 2 entries:
>>> Offset Info Type Sym. Value Sym.
>>> Name
>>> + Addend
>>> 000000000020 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
>>> 000000000040 000200000002 R_X86_64_PC32 0000000000000000 .text + 7
>>>
>>> Relocation section '.rela.sframe' at offset 0xaa88 contains 2 entries:
>>> Offset Info Type Sym. Value Sym.
>>> Name
>>> + Addend
>>> 00000000001c 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
>>> 000000000030 000200000002 R_X86_64_PC32 0000000000000000 .text + 7
>>
>> In this context it would help if the corresponding source was also shown.
temp.s could be, for example,
.text
.type foo, @function
foo:
.cfi_startproc
push %rbp
.cfi_def_cfa_offset 16
.cfi_offset 6, -16
mov %rsp,%rbp
.cfi_def_cfa_register 6
nop
pop %rbp
.cfi_def_cfa 7, 8
ret
.cfi_endproc
.size foo, .-foo
.type bar, @function
bar:
.cfi_startproc
nop
ret
.cfi_endproc
.size bar, .-bar
$ objdump -d temp.o
...
0000000000000000 <foo>:
0: 55 push %rbp
1: 48 89 e5 mov %rsp,%rbp
4: 90 nop
5: 5d pop %rbp
6: c3 ret
0000000000000007 <bar>:
7: 90 nop
8: c3 ret
>> Without that it's hard to see why this is incorrect, while ...
>>
Jens summarizes the issue better than I can...
>>> But then, ld.bfd is then fixing up the relocated value of
>>> sfde_func_start_address by adding the r_offset (in
>>> _bfd_elf_merge_section_sframe ()). So finally what ld.bfd currently
>>> generates
>>> (for E_DYN and E_EXEC) is aligning to Scheme #A. Thinko it was, but
>>> moving
>>> forward, this needs to be fixed because this essentially means we are
>>> imposing
>>> custom semantics of an existing PC-rel RELA relocation for E_REL
>>> binaries.
>>> https://lore.kernel.org/linux-arm-kernel/20250225230137.620606-1-
>>> wnliu@google.com/T/#m1751e4abf1186ab2ae9064478d453d48542fe063
>>>
>>> * The fix: Use Scheme #A consistently
>>>
>>> If we fix the thinko in GAS described above, we now build the expression
>>> `.text + text_offset - .sframe'. This expression results in GAS
>>> emitting a PC-rel relocation such as:
>>>
>>> Relocated value = .text + text_offset - .sframe
>>>
>>> (S) .text |
>>> |
>>> -----------> | function foo \
>>> | | |
>>> | | | .text + text_offset - .sframe
>>> | | |
>>> | | /
>>> ---> .sframe | \
>>> | | P (sframe_offset)
>>> | FDE (start_func_addr) /
>>>
>>> Where:
>>>
>>> S = .text
>>> A = text_offset + sframe_offset
>>> P = (.sframe +) sframe_offset
>>>
>>> Note how the addend A carries the sframe_offset (because it is the
>>> r_offset of
>>> the RELA) only to be annulled by substracting sframe_offset again when
>>> calculating the relocation with S + A - P. This is GAS's way to
>>> encode this
>>> "offset between two points one of which is the section containing the
>>> PC/relocated-location". While the calculation is correct, it leads to
>>> complications in the linker when linking to generate relocatable
>>> objects, as
>>> explained later below.
>>>
>>> So with the fixed GAS, we will see the following:
>>>
>>> $ readelf -r temp.o
>>> ...
>>> Relocation section '.rela.eh_frame' at offset 0xaa58 contains 2 entries:
>>> Offset Info Type Sym. Value Sym.
>>> Name
>>> + Addend
>>> 000000000020 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
>>> 000000000040 000200000002 R_X86_64_PC32 0000000000000000 .text + 7
>>>
>>> Relocation section '.rela.sframe' at offset 0xaa88 contains 2 entries:
>>> Offset Info Type Sym. Value Sym.
>>> Name
>>> + Addend
>>> 00000000001c 000200000002 R_X86_64_PC32 0000000000000000 .text + 1c
>>> 000000000030 000200000002 R_X86_64_PC32 0000000000000000 .text + 37
>>
>> ... this would be correct. Furthermore ...
>>
>>> IMP: Note how the r_addend of the reloc in .rela.sframe is updated (and
>>> different compared to the r_addends shown for .rela.eh_frame) by
>>> including the
>>> r_offset in it. The actual "text_offset" is now merged with the
>>> "r_offset".
>>> This brings in link-time considerations as we will see later.
>>>
>>> * Linking Considerations with the fix
>>>
>>> To set the context for the linking process, note how information is
>>> organized
>>> in SFrame sections:
>>>
>>> --------------------- ---------------------
>>> | SFrame Header | | SFrame Header |
>>> --------------------- ---------------------
>>> | SFrame FDE 1a | (Linking) | SFrame FDE 1a |
>>> | SFrame FDE 1b | | SFrame FDE 1b |
>>> | ... | |------>| ... |
>>> | SFrame FDE 1n | | | SFrame FDE 1n |
>>> --------------------- | | SFrame FDE 2a |
>>> | ... | ------ | SFrame FDE 2b |
>>> |SFrame FREs (Sec 1)| | | ... |
>>> |(Frame Row Entries)| | | SFrame FDE 2n |
>>> | ... | | |-------------------|
>>> | ... | | | ... |
>>> --------------------- | |SFrame FREs (Sec 1)|
>>> | |SFrame FREs (Sec 2)|
>>> | |(Frame Row Entries)|
>>> | | ... |
>>> --------------------- | ---------------------
>>> | SFrame Header | |
>>> --------------------- |
>>> | SFrame FDE 2a | |
>>> | SFrame FDE 2b | |
>>> | ... | ------
>>> | SFrame FDE 2n |
>>> ---------------------
>>> | ... |
>>> |SFrame FREs (Sec 2)|
>>> |(Frame Row Entries)|
>>> | ... |
>>> | ... |
>>> ---------------------
>>>
>>> Figure 1 SFrame information layout and linking process
>>>
>>> Unlike EH_Frame, where a DWARF FDE (containing its associated DWARF
>>> instructions) is relocated as a single unit, SFrame FDEs and FREs are
>>> grouped
>>> separately. A function's stack trace data requires one SFrame FDE and
>>> multiple
>>> SFrame FREs. This structure impacts how ld.bfd outputs RELAs for SFrame
>>> sections during relocatable links.
>>>
>>> * Relocatable Links and r_addend Update
>>>
>>> With the fixed GAS following Scheme #A, a PC-rel relocation is
>>> generated to
>>> (correctly) express the offset between the PC and the section
>>> containing the
>>> relocated-location. This is achieved by adding r_offset (P) to A, and
>>> when the
>>> relocation is resolved (S + A - P), it effectively cancels out P,
>>> yielding the
>>> desired result.
>>>
>>> This, however, has implications for relocatable links: we need to
>>> manually
>>> calculate the r_addend for the output RELAs. We now have to manually
>>> "extract"
>>> the "text_offset" from the input-reloc's A. We then need to add to it
>>> the new
>>> r_offset of the SFrame FDE in the output SFrame section to finally
>>> obtain the A
>>> for the output reloc.
>>
>> ... such custom handling shouldn't normally be required. Everything would
>> better come out "natural".
>
> IIUC a PC-relative relocation is expected to be relative to the location
> of its associated field. In Scheme #A it is used (for the FDE function
> start address - think of it as function start offset) as if it was
> relative to the section its associated field resides in. This causes
> the offset of the associated field from its section to be baked into the
> addend.
>
> Merging of SFrame sections gathers all FDEs from the input sections and
> places them together in the output section. In any case this requires
> to update the relocation r_offset, as the location of the field changes.
> For Scheme #A this also causes the baked-in field offset from section in
> the addend to change, making it necessary to update the addend.
>
> @Indu:
> Sorting of the FDEs (based on FDE function start address) is also
> performed for relocatable links. IIUC it is effectively skipped, as
> the (non-relocated) FDE function start address is zero for all FDEs,
> so that fde_func() always compares equal and qsort() does nothing in
> sframe_sort_funcdesc().
> Wouldn't it make sense to explicitly skip sorting of FDEs for
> relocatable links? I also wonder whether the FDEs in a relocatable
> link are really always sorted. It appears to be true for single output
> .text section, as input .text sections then get concatenated in the
> order they appear as input (unless there is some way to change this via
> linker script?). But if there are multiple .text sections (e.g. .text
> and .text.hot) the output does not look right for relocatable links.
>
>>> This additional logic is carved out as a separate patch "[RFC 3/4]
>>> ld: bfd:
>>> sframe: fix incorrect r_offset in RELA entries" to aid discussion. As
>>> you see
>>> this is being done in generic linker code in elf_link_input_bfd ().
>>>
>>> * The r_addend Update Issue
>>>
>>> If such a "r_addend fixup" is risky or wrong, it seems we will need a
>>> new type
>>> of RELOC for SFrame sections ? This is the reason the series is marked
>>> as RFC.
>>> I am not completely clear on whether doing so has impact on current/
>>> future
>>> workflows in ld.bfd.
>>>
>>> * Miscellaneous
>>>
>>> The update of irela->r_offset in elf_link_input_bfd() is necessary,
>>> irrespective of scheme #A or #B, as explained in "[RFC 3/4] ld: bfd:
>>> sframe: fix
>>> incorrect r_offset in RELA entries".
>>>
>>> As depicted in Figure 1, the offset of an SFrame FDE in the output
>>> section is
>>> not simply "input_section->output_offset + r_offset". Therefore, the
>>> following
>>> explicit stub is required in elf_link_input_bfd():
>>>
>>> if (o->sec_info_type != SEC_INFO_TYPE_SFRAME)
>>> irela->r_offset += o->output_offset;
>>>
>>> IOW, "[RFC 3/4] ld: bfd: sframe: fix incorrect r_offset in RELA
>>> entries" is
>>> essential to resolve PR 32666 and enable relocatable links, regardless
>>> of the
>>> chosen scheme. I include it in the series as I need the relocatable
>>> links to
>>> work to later show the r_added related stubs.
>
> Regards,
> Jens
More information about the Binutils
mailing list