[RFC 2/5] elf: Add SFrame stack tracing
Claudiu Zissulescu-Ianculescu
claudiu.zissulescu-ianculescu@oracle.com
Tue Mar 25 11:26:20 GMT 2025
>> +/* Initialize the SFrame backtrace routine, and try to backtrace the
>> + current stack using SFrame info. The return value of SFrame stack
>> + tracer must be larger than one to consider the SFrame backtrace
>> + valid. Otherwise, there may be the case that glibc is compiled
>> + using SFrame but the application not. */
>
> This is because DO_SFRAME_BACKTRACE() then managed to find SFrame
> information for __backtrace() as the topmost and only frame in Glibc
> and at least one further function from the application, right?
Right.
>
>> +
>> +#define DO_SFRAME_BACKTRACE(ARRAY, SIZE) \
>> + do \
>> + { \
>> + int cnt; \
>> + frame frame; \
>> + frame.pc = getPC(); \
>> + frame.sp = getSP(); \
>> + frame.fp = (_Unwind_Ptr) __builtin_frame_address (0); \
>
> Why doesn't this require to be wrapped in a noinline helper getFP()? If
> there is some clever trickery involved , could you maybe add a comment?
If I will do so, then I don't obtain the FP of the current function
(i.e., __bactrace()) but the FP of the wrapper. For SFrame intial
conditions, I need the current RA, SP, and FP. For RA, I use
return_address builtin. For SP, I use dwarf_cfa in a wrapper, but
starting with gcc14, I can use __builtin_stack_address() without any
wrapping. However, for portability, it is better to abuse dwarf_cfa builtin.
>
>> + cnt = stacktrace_sframe (ARRAY, SIZE, &frame); \
>> + if (cnt > 1) \
>> + return cnt; \
>> + } \
>> + while(0)
>> +
>> +static _Unwind_Ptr __attribute__ ((noinline))
>> +getPC (void)
>> +{
>> + return (_Unwind_Ptr) __builtin_return_address (0);
>
> Clever to wrap this in a noinline helper, to use this builtin to obtain
> the PC in __backchain() (at invocation of the helper).
The same mechanism is used by DWARF unwinder too :)
>
> Although not required on x86-64, AArch64, and s390x: Would it make sense
> to use __builtin_extract_return_addr() in addition?
I don't see any harm. There are a handful of arches requiring extract
return address builtin, but they are not supporting SFrame afaik.
>
>> +}
>> +
>> +static _Unwind_Ptr __attribute__ ((noinline))
>> +getSP (void)
>> +{
>> + return (_Unwind_Ptr) __builtin_dwarf_cfa();
>
> This only works for architectures that define their CFA as SP at
> call site. On s390x the CFA is defined as SP at call site +160.
> I will have to make either this helber architecture dependent or
> introduce an architecture dependent SP value offset from CFA (e.g.
> SFRAME_SP_VAL_OFFSET).
I am not familiar with s390x, but for those arches, maybe we need to add
a cpu specific hook when the time is right.
>
>> +}
>> +
>> static _Unwind_Reason_Code
>> backtrace_helper (struct _Unwind_Context *ctx, void *a)
>> {
>> @@ -72,7 +105,13 @@ __backtrace (void **array, int size)
>> .cnt = -1
>> };
>> - if (size <= 0 || arg.unwind_link == NULL)
>> + if (size <= 0)
>> + return 0;
>> +
>
> It's kind of obvious, but maybe add the following comment for those that
> do not know SFrame:
>
> /* Thy the SFrame stack tracer. */
Will do.
>
>> + DO_SFRAME_BACKTRACE (array, size);
>> +
>> + /* Try the dwarf unwinder. */
>> + if (arg.unwind_link == NULL)
>> return 0;
>> UNWIND_LINK_PTR (arg.unwind_link, _Unwind_Backtrace)
>
>> diff --git a/sysdeps/generic/sframe.c b/sysdeps/generic/sframe.c
>
>> @@ -0,0 +1,140 @@
>> +/* Copyright (C) 2025 Free Software Foundation, Inc.
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public License as
>> + published by the Free Software Foundation; either version 2.1 of the
>> + License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU General Public License
>> + along with this program. If not, see <http://www.gnu.org/
>> licenses/>. */
>> +
>> +#include <sframe-read.h>
>> +#include <stdlib.h>
>> +#include <dlfcn.h>
>> +#include <unwind.h>
>> +#include <uw-sigframe.h>
>> +
>> +#if DLFO_STRUCT_HAS_SFRAME
>
> IIUC this makes Glibc backtrace() use of SFrame for stack tracing solely
> depend on whether an architecture has enabled DLFO_STRUCT_HAS_SFRAME.
> Both x86-64 and AArch64 enable this unconditionally with your subsequent
> patches.
>
> Wouldn't it make sense to not enable it if Glibc is not compiled with
> SFrame stack trace information? Otherwise __backtrace() would always
> attempt to use SFrame first with stacktrace_sframe() failing to locate
> any SFrame information for __backtrace() itself.
The SFrame stack tracing can be disable in glibc by using
--disable-sframe while configuring. By all means, the SFrame feature is
not enabled unless toolchain supports sframe. This check is done during
configuration phase.
The idea here is to try first with SFrame, and then using the DW unwinder.
>
> In addition I wonder whether it would be good to allow to build Glibc
> with SFrame stack trace information but disable the use in __backtrace().
Good question, It may imply an extra configuration option, but what it
will be the added value? The reason of using SFrame for stack tracing is
the format its self as it lighter than DW.
>
>> +
>> +static inline uint64_t
>> +read_stack_value (uint64_t loc)
>> +{
>> + uint64_t value = *((uint64_t *) loc);
>> + return value;
>> +}
>> +
>> +/* Backtrace the stack and collect the stacktrace given SFrame info.
>> + If successful, store the return addresses in RA_LST. The SIZE
>> + argument specifies the maximum number of return addresses that can
>> + be stored in RA_LST and contains the number of the addresses
>> + collected. */
>> +
>> +int
>> +stacktrace_sframe (void **ra_lst, int count, frame *frame)
>> +{
>> + _Unwind_Ptr sframe_vma, cfa, return_addr, ra_stack_loc,
>> rfp_stack_loc, pc;
>> + int cfa_offset, rfp_offset, ra_offset, i;
>
> Why rfp_stack_loc and rfp_offset instead of fp_stack_loc and fp_offset?
Just naming, I will correct them.
>
>> + sframe_frame_row_entry fred, *frep = &fred;
>> +
>> + if (!ra_lst || !count)
>> + return 0;
>> +
>> + for (i = 0; i < count; i++)
>> + {
>> + _Unwind_Reason_Code err;
>> + struct dl_find_object data;
>> + sframe_decoder_ctx decoder_context, *dctx = &decoder_context;
>> +
>> + /* Clean decoder context. */
>> + memset (dctx, 0, sizeof (sframe_decoder_ctx));
>> +
>> + /* Load and set up the SFrame stack trace info for pc. */
>> + if (_dl_find_object ((void *) frame->pc, &data) < 0)
>> + return i;
>> +
>> + sframe_vma = (_Unwind_Ptr) data.dlfo_sframe;
>> + if (!sframe_vma)
>> + {
>> +#ifdef MD_DECODE_SIGNAL_FRAME
>> + /* I cannot find a valid SFrame section. Check if it is a
>> + signal frame. */
>> + if (MD_DECODE_SIGNAL_FRAME (frame) == _URC_NO_REASON)
>> + {
>> + ra_lst[i] = (void *) frame->pc;
>> + continue;
>> + }
>> +#endif
>> + return i;
>> + }
>> +
>> + /* Decode the specified SFrame buffer populate sframe's decoder
>> + context. */
>> + if (sframe_decode (dctx, (char *) data.dlfo_sframe) ==
>> _URC_END_OF_STACK)
>> + return i;
>> +
>> + pc = frame->pc - sframe_vma;
>> + /* Find the SFrame Row Entry which contains the PC. */
>> + if (sframe_find_fre (dctx, pc, frep) == _URC_END_OF_STACK)
>> + {
>> +#ifdef MD_DECODE_SIGNAL_FRAME
>> + /* I cannot find any FREs, try to see if it is a signal
>> + frame, and if so decode it. */
>> + if (MD_DECODE_SIGNAL_FRAME (frame) == _URC_NO_REASON)
>> + {
>> + ra_lst[i] = (void *) frame->pc;
>> + continue;
>> + }
>> +#endif
>> + return i;
>> + }
>> +
>> + /* Get the CFA offset from the FRE. */
>> + cfa_offset = sframe_fre_get_cfa_offset (dctx, frep);
>> +
>> + /* Get the base reg id from the FRE info. */
>
> The comment is misleading, as the computed value is the CFA. Maybe:
>
> /* Get CFA using base reg id from the FRE info. */
Noted
>
>> + cfa = ((sframe_fre_get_base_reg_id (frep)
>> + == SFRAME_BASE_REG_SP) ? frame->sp : frame->fp) + cfa_offset;
>> +
>> + /* Get the RA offset from the FRE. */
>> + ra_offset = sframe_fre_get_ra_offset (dctx, frep);
>> +
>> + ra_stack_loc = cfa + ra_offset;
>> + return_addr = read_stack_value (ra_stack_loc);
>
> On s390x the RA (and FP) may be saved in registers in the topost frame,
> e.g. when in a leaf function. In that case s390x will encode the DWARF
> register number in the SFrame RA/FP offset. The good thing is that
> neither the caller of backtrace() nor __backtrace() itself are leaf
> functions, so that this special case may not occur in the this Glibc
> use case. I would need to add a check though, that the returned offset
> is not an encoded DWARF register number on s390x.
Probably there are a number of hooks which needs to be defined for more
exotic arches. When do you forsee s390x to land in SFrame specs?
>
>> +
>> + ra_lst[i] = (void *)return_addr;
>> +
>> + /* Set up for the next frame. */
>> + /* Get the FP offset from the FRE. If the offset is invalid,
>> + sets errp. */
>> + rfp_offset = sframe_fre_get_fp_offset (dctx, frep, &err);
>> + if (err == _URC_NO_REASON)
>> + {
>> + /* Frame is valid, get the value stored in the stack
>> + location. */
>> + rfp_stack_loc = cfa + rfp_offset;
>> + frame->fp = read_stack_value (rfp_stack_loc);
>
> Same as for RA above.
>
>> + }
>> +
>> + frame->sp = cfa;
>
> Same as for GetSP() above. I guess introducing an architecture
> dependent SP value offset from CFA (e.g. SFRAME_SP_VAL_OFFSET) would be
> the easiest solution? On architectures, such as x86-64 and AArch64,
> where it would be defined to 0 the compiler would optimize the offset
> away. What do you think?
I am trying to stay away from complicating the code :) It is difficult
for me to defend something which I am not aware of. However, in the
moment when s390x sframe support is added, probably, it will be easy
defended by the authors.
>
>> + frame->pc = return_addr;
>> + }
>> + return i;
>> +}
>> +#else
>
> #else /* !DLFO_STRUCT_HAS_SFRAME */
>
>> +/* Dummy function called when SFrame is not available for a target. */
>> +int
>> +stacktrace_sframe (void **ra_lst __attribute__ ((__unused__)),
>> + int count __attribute__ ((__unused__)),
>> + frame *frame __attribute__ ((__unused__)))
>> +{
>> + return 0;
>> +}
>> +#endif /* DLFO_STRUCT_HAS_SFRAME */
>> +
>> +libc_hidden_def (stacktrace_sframe)
>
>> diff --git a/sysdeps/generic/sframe.h b/sysdeps/generic/sframe.h
>
>> @@ -0,0 +1,359 @@
>> +/* SFrame format description.
>> + Copyright (C) 2022-2025 Free Software Foundation, Inc.
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU General Public License
>> + along with this program; see the file COPYING. If not see
>> + <http://www.gnu.org/licenses/>. */
>
> Would it make sense to add a note that this is a copy of GNU Binutils
> include/sframe.h with only minor adaptions and additions and therefore
> should be kept in sync?
IMHO, we should not do that, as binutils can evolve much faster and add
v3 or other features, while this implementation may evolve on a
different step and have its own dev life.
>
>> +
>> +#ifndef _SFRAME_H
>> +#define _SFRAME_H
>
> Thanks and regards,
> Jens
More information about the Libc-alpha
mailing list