[RFC 15/28] [SFrame-V3] include: sframe: reserve 4 bits for new FDE types

[Indu Bhagat]

On 12/12/25 9:05 AM, Tatsuyuki Ishi wrote:
>> On Dec 12, 2025, at 19:20, Indu Bhagat via Binutils 
>> <binutils@sourceware.org> wrote:
>>
>> On 12/11/25 9:36 AM, Jens Remus wrote:
>>> Hello Indu,
>>> I am trying to wrap my head around the new FLEX_TOPMOST_FRAME encoding.
>>
>> (Adding Sterling, Pavel who are working on the LLVM support to the 
>> thread too)
>>
>>> On 12/9/2025 10:07 AM, Indu Bhagat wrote:
>>>> In SFrame V2, the FDE representation caters to the most common cases of
>>>> stack trace metadata:
>>>>   - CFA is SP/FP based,
>>>>   - FP/RA offsets, if present are CFA based (except some cases in s390x
>>>>     where reg can be encoded).
>>>>
>>>> Introduce an additional FDE type SFRAME_FDE_TYPE_FLEX_TOPMOST_FRAME,
>>>> which can encode a more flexible set of CFA, FP and RA recovery rules.
>>>> Some of the patterns supported include:
>>>>   - CFA may be non-SP/FP based.
>>>>   - CFA, FP may encode dereferencing of register after offset adjustment
>>>>   - RA may be in a non-default register.
> 
> For DRAP, interestingly, "CFA may be non-SP/FP based” only applies 
> during the prolog, where you get something like:
>      CFA=R10: RBP=[DW_OP_breg6 RBP+0], RIP=[CFA-8]
> 
> However, in function body CFA & FP needs to encode dereferencing of 
> register, and FP is not an offset from CFA:
>      CFA=DW_OP_breg6 RBP-40, DW_OP_deref: RBP=[DW_OP_breg6 RBP+0]
> 

Right.

To cater to these sequences, SFRAME_FDE_TYPE_FLEX_TOPMOST_FRAME 
explicitly encodes the de-referencing of a location.  FWIW, ORC stack 
trace format in the Kernel uses the same strategy:
   - For DRAP, encode the register (which is used as the base for CFA) 
in the stack trace data
   - Use the register for topmost frame.

WRT SFrame V3, using SFRAME_FDE_TYPE_FLEX_TOPMOST_FRAME, we will encode 
the above as:
   CFA=r10
   rbp=*(rbp)
   ra=*(cfa-8)

and for the function body:
   CFA=*(rbp-40)
   rbp=*(rbp)

Where (*) denotes the dereferencing.

> You can see a full sample dump in https://gist.github.com/ 
> ishitatsuyuki/0dc1845118c4c8b636b0d16e24065875 <https://gist.github.com/ 
> ishitatsuyuki/0dc1845118c4c8b636b0d16e24065875>
> 

Thanks. This matches with my observations.

>>>>
>>>> The important bit here is that since SFrame does not track all
>>>> callee-saved registers, the above-mentioned recovery rules must only be
>>>> done for top-most frames (by the stack tracer).
>>>>
>>>> Adding a new FDE type does have implications for a stack tracer in that
>>>> it needs to:
>>>>   - Check the FDE type before interpreting the Stack offsets data.
>>>>   - If the FDE type is SFRAME_FDE_TYPE_FLEX_TOPMOST_FRAME, and the
>>>>     recovery rules employ the use of any non-SP/FP based register, the
>>>>     stack tracer must proceed only if it is the topmost frame on stack.
>>> Can this be summarize to:  Recovery rules that employ arbitrary register
>>> contents (other than SP/FP) are only allowed in the topmost frame?
>>
>> Yes.
> 
> Are we gating both non-SP/FP registers and complex dereferencing behind 
> TOPMOST_FRAME?
> The complex dereferencing can be needed even in synchronous unwinding, 
> like the example above.
> 
> If it’s too costly, we could drop support for non-SP/FP registers, since 
> we only hit them if unwinding a function with DRAP during the prolog.
> Functions with DRAP are rare, and the use cases I’ve heard so far can 
> tolerate unwind failures as long as they are clearly indicated.
> I’m leaning toward the conclusion that getting 100% coverage is 
> impractical, and some userspace JIT impls have rare cases where unwind
> is simply not possible (e.g. when manipulating stack during deoptimization).
> 

True, I agree that 100% coverage gets to a point of high overhead-low 
returns pretty quickly at some point.  We may even get asymptotically 
close to 100% for some ABIs, but in principle, it should be OK to leave 
some bits as unrepresented.

This is subjective, but: support for DRAP pattern hovers somewhere in 
that zone too.

Anyway, the positive aspect of encoding a distinct FDE type now 
(SFRAME_FDE_TYPE_FLEX_TOPMOST_FRAME) is that it allows for a provision 
in the format now to represent these type of corner cases.

The important bit is also that: For supporting the DRAP pattern, we do 
not start tracking these additional registers (say, r10, rcx..) in 
SFrame.  We instead rely on stack tracers to not pursue the information 
in SFRAME_FDE_TYPE_FLEX_TOPMOST_FRAME if it is not the topmost frame.

>>
>>>>
>>>> For each of CFA and RA two offsets are always used.  The two offsets
>>>> represent the information as follows:
>>>>   - (minimum 8-bit) offset1 to encode register like:
>>>>     (regnum << 3) | unused << 2 | deref << 1 | reg_p (=1)
>>>>   - offset2 to encode offset: offset
>>>>
>>>> reg_p = 1 indicates a register, reg_p = 0 indicates CFA.
>>>>
>>>> The offsets are in the usual order:  CFA, RA, FP if present.
>>> Is it required to have this flexibility for both CFA and RA/FP?  Would
>>> it be an alternative to use the common encoding but leave some features
>>> as "reserved" in each?
>>
>> For DRAP, it is required to have this sort of flexibility for CFA and 
>> FP.  For RA, there are no known patterns yet requiring flexibility.
> 
> In DRAP RA seems to be always at [CFA-8], although a custom RA is still 
> useful for signal frames or similar patterns
> (e.g. Wine usage I mentioned in https://sourceware.org/pipermail/ 
> binutils/2025-November/145523.html <https://sourceware.org/pipermail/ 
> binutils/2025-November/145523.html>).
> 
> For the reference, this is how signal frames look like:
>      CFA=DW_OP_breg7 RSP+160, DW_OP_deref: RAX=[DW_OP_breg7 RSP+144], 
> RDX=[DW_OP_breg7 RSP+136], RCX=[DW_OP_breg7 RSP+152], RBX=[DW_OP_breg7 
> RSP+128], RSI=[DW_OP_breg7 RSP+112], RDI=[DW_OP_breg7 RSP+104], 
> RBP=[DW_OP_breg7 RSP+120], RSP=[DW_OP_breg7 RSP+160], R8=[DW_OP_breg7 
> RSP+40], R9=[DW_OP_breg7 RSP+48], R
> 10=[DW_OP_breg7 RSP+56], R11=[DW_OP_breg7 RSP+64], R12=[DW_OP_breg7 
> RSP+72], R13=[DW_OP_breg7 RSP+80], R14=[DW_OP_breg7 RSP+88], 
> R15=[DW_OP_breg7 RSP+96], RIP
> =[DW_OP_breg7 RSP+168]
> And Wine’s PE/Unix trampoline:
>      CFA=DW_OP_breg6 RBP-16, DW_OP_deref: RBX=[DW_OP_breg6 RBP-144], 
> RSI=[DW_OP_breg6 RBP-120], RDI=[DW_OP_breg6 RBP-112], RBP=[DW_OP_breg6 
> RBP+0], R12=[DW_OP_breg6 RBP-72], R13=[DW_OP_breg6 RBP-64], 
> R14=[DW_OP_breg6 RBP-56], R15=[DW_OP_breg6 RBP-48], RIP=[DW_OP_breg6 RBP-40]
> 
> FLEX_TOPMOST_FRAME can cover these, but these frames can appear anywhere 
> in the middle of a stack.
> 

Right, the signal frame trampoline could be the non-topmost frame.

For the signal trampolines above, the tricky part is the rule for RSP in 
the CFI opcodes for the AMD64/Linux.  The intended effect of these is:
   - CFA = *(rsp+160)
   - rbp = *(rsp+120)
   - rsp = *(rsp+160)
   - RA = *(rsp+168)

Note that in SFrame, we cannot explicitly encode information to restore 
stack-pointer.  Restoration of stack-pointer is based on implicit rules 
(CFA).  But IIUC, in the above rules, the implicit rule of CFA = sp (at 
__restore_rt boundary) is not violated. (right?)

> I haven’t had full time to read through the full code, so apologies if I 
> missed something.
> 
> Tatsuyuki Ishi
> >>
>> I kept some notes and I just published them here
>>
>> https://github.com/ibhagatgnu/work-notes/blob/main/sframe/flexible- 
>> topmost-frame/flexible-sframe-fde.rst <https://github.com/ibhagatgnu/ 
>> work-notes/blob/main/sframe/flexible-topmost-frame/flexible-sframe- 
>> fde.rst>
>>
>>>> For example, for FP/RA tracking,
>>>>
>>>> a) If the reg is REG1 for FP/RA tracking,
>>>>    - Encoding:
>>>>      + offset1 to encode register: (REG1 << 3) | unused << 2 | deref 
>>>> << 1 | reg_p (=1)
>>>>      + offset2 to encode offset: offset
>>>>    - Action:
>>>>      + if deref, FP/RA = \*(REG1 + offset) (e.g., seen for FP recovery
>>>>        with certain DRAP patterns on x86_64)
>>> How can this be expressed by CFI directives?  Could you please provide
>>> an example?
>>
>> The document above should have this info.
>>
>>>>      + if no deref, FP/RA = REG1 + offset
>>> Should s390x use that to encode FP/RA saved in register?  Since that is
>>> only supported for topmost frames FLEX_TOPMOST_FRAME seems a good fit.
>>> It would remove the need for a s390x-specific encoding and would also
>>> make your patch "[RFC 13/28] [SFrame-V3] include: sframe: s390x: keep
>>> unused bits when possible" obsolete.
>>
>> The disadvantage to doing this would be : the representation for 
>> FLEX_TOPMOST_FRAME is a bit bloated.
>>
>> We can consider it if people think this is a good idea.  I will think 
>> about it.
>>
>>> It would be interesting to see the .sframe section size penalty though.
>>> I would need to implement a prototype to compare based on your patch
>>> "[RFC 21/28] [SFrame-V3] gas: sframe: add handling for .cfi_register
>>> for FP and RA".  Have you had considered this?  If yes why didn't you
>>> decide for that approach?
>>
>> Umm, not sure I understand this completely.  Can you reframe the 
>> question ?
>>
>> So far, I did some correctness checks on gcc package builds with 
>> SFrame enabled.  I didnt save the section sizes of enabling 
>> FLEX_TOPMOST_FRAME.  Will get back to you.
>>
>>>> b) If the reg is CFA for FP/RA tracking,
>>>>    - Encoding:
>>>>      + [=Effectively Padding] offset1 to encode register:
>>>>         (( 0 << 3 ) | unused << 2 | 0 << 1 | reg_p (=0))
>>> Nit:
>>>         (( 0 << 3 ) | unused << 2 | deref << 1 | reg_p (=0))
>>>>      + offset2 to encode offset: offset
>>>>    - Action:
>>>>      + if deref, FP/RA = *(CFA + offset)
>>> Makes sense.
>>>>      + if no deref, FP/RA = CFA + offset (pattern shouldnt be seen for
>>>>        RA)
>>> This could be used to represent .cfi_val_offset with FP/RA register,
>>> correct?  Although I have only ever seen .cfi_val_offset with
>>> SP register so far.
>>
>> Yes, it could be used.  Although I havent implemented it yet due to 
>> their low occurrence.
>>
>>>> Next for CFA tracking,
>>>>    - Action:
>>>>      + if deref, CFA = *(reg + offset) (e.g., seen for CFA recovery in
>>>>        some stack realignment patterns on AMD64)
>>> How can this be expressed by CFI directives?  Could you please provide
>>> an example?
>>
>> The doc has example.
>>
>>>>      + if no deref, CFA = reg + offset (e.g., for .cfi_def_cfa 2, 8, or
>>>>        .cfi_def_cfa 10, 0)
>>> This is used to represent .cfi_def_cfa[_offset|_register] with non-SP/FP
>>> register, as in the following example, correct?
>>> .cfi_def_cfa <non-SP/FP-reg>, <offset>
>>> Is encoded as:
>>> offset1 = (regnum=<non-SP/FP-reg>) << 3 | (deref=0) << 1 | (reg_p=1)
>>> offset2 = <offset>
>>>>
>>>> Expected usage of this FDE type is quite low (DRAP on x86_64).
>>> So a FLEX_TOPMOST_FRAME would encode SP/FP-based CFA using the register
>>> number instead of the FRE use_fp flag?  This then requires knowledge of
>>> the SP and FP DWARF register numbers in the stacktracer, so that it can
>>> decide whether a rule is allowed in a non-topmost frame.  In the Kernel
>>> I would hide those in unwind user sframe, so that the common unwind user
>>> framework does not need to have this knowledge (see SP_OFFSET and
>>> FP_OFFSET CFA rules below).
>>
>> Yes, stack tracer now requires the knowledge of SP/FP and also other 
>> DWARF register numbers.  The offset1 in FLEX_TOPMOST_FRAME encodes the 
>> DWARF register number when applicable.
>>
>>> To support a most of the new FLEX_TOPMOST_FRAME encoding in the Kernel
>>> common unwind user framework, I roughly assume the following high-level
>>> changes:
>>
>> I took one look at the changes below.  I will revisit tomorrow 
>> morning. I made some notes in the document, but they are at the pseudo 
>> code level.
>>
>>> For the CFA:
>>> +enum unwind_user_cfa_rule_type {
>>> +UNWIND_USER_CFA_RT_SP_OFFSET,// CFA = SP + 
>>> offset(formerly .use_fp=false)
>>> +UNWIND_USER_CFA_RT_FP_OFFSET,// CFA = FP + offset(formerly .use_fp=true)
>>> +UNWIND_USER_CFA_RT_REG_OFFSET,// CFA = <non-SP/FP register> + offset
>>> +UNWIND_USER_CFA_RT_REG_OFFSET_DEREF,// CFA = *(<non-SP/FP register> 
>>> + offset)
>>> +};
>>> +
>>> +struct unwind_user_cfa_rule {
>>> +enum unwind_user_cfa_rule_type type;
>>> +s32 offset;
>>> +int regnum;
>>> +};
>>> Note that I omitted the following exotic cases.  Not sure whether they
>>> need to be supported?
>>> UNWIND_USER_CFA_RT_SP_OFFSET_DEREF,// CFA = *(SP + offset)
>>> UNWIND_USER_CFA_RT_FP_OFFSET_DEREF,// CFA = *(FP + offset)
>>> The SP_OFFSET and FP_OFFSET cases hide the need for unwind user to know
>>> their architecture-specific DWARF register numbers.  unwind user sframe
>>> would need to know them, depending on whether SFrame V3 still uses the
>>> FDE use_fp flag.
>>> For FP/RA:
>>>  enum unwind_user_rule_type {
>>> UNWIND_USER_RT_UNKNOWN,// FP/RA = 0 (used with unwind user fp on s390 
>>> using back chain)
>>> UNWIND_USER_RT_RETAIN,// FP/RA = FP/RA
>>> -UNWIND_USER_RT_STACK,
>>> -UNWIND_USER_RT_REG,
>>> +UNWIND_USER_RT_CFA_OFFSET_DEREF,// FP/RA = *(CFA + offset)(formerly 
>>> UNWIND_USER_LOC_STACK)
>>> +UNWIND_USER_RT_REG_OFFSET,// FP/RA = register + offset(formerly 
>>> UNWIND_USER_LOC_REG)
>>> +UNWIND_USER_RT_REG_OFFSET_DEREF,// FP/RA = *(register + offset)
>>>  };
>>>  struct unwind_user_rule {
>>> enum unwind_user_rule_type type;
>>> -union {
>>> -s32 offset;
>>> -int regnum;
>>> -};
>>> +s32 offset;
>>> +int regnum;
>>>  };
>>> Note that the following omitted case would be required for
>>> .cfi_val_offset <FP/RA> and could easily be added:
>>> UNWIND_USER_RT_CFA_OFFSET,// FP/RA = CFA + offset
>>>  struct unwind_user_frame {
>>> -s32 cfa_off;// now .cfa.offset
>>> -s32 sp_off;// moved
>>> +struct unwind_user_cfa_rule cfa;
>>> struct unwind_user_rule ra;
>>> struct unwind_user_rule fp;
>>> -bool use_fp;// now .cfa.type = SP_OFFSET/FP_OFFSET 
>>> (formerly .use_fp=false/true)
>>> +s32 sp_off;
>>> bool outermost;
>>>  };
>>> Note that rule types that contain "_REG_" are only allowed in the
>>> topmost frame.
>>> Maybe the _DEREF cases could better be represented as follows, but that
>>> would allow all kinds of dereference variations that are not sensible,
>>> e.g. UNKNOWN or RETAIN with deref=true.
>>> struct unwind_user_[cfa_]rule {
>>> ...
>>> bool deref;
>>> };
>>> Thoughts?
>>> Thanks and regards,
>>> Jens
>