libc with ARM unwind tables and frame pointers

Sundar Dev sundarjdevml@gmail.com
Fri Jun 26 22:18:00 GMT 2015 

Hi Adhemerval:

On Fri, Jun 26, 2015 at 12:57 PM, Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
> Hi
>
> On 26-06-2015 15:26, Sundar Dev wrote:
>> Hi List Members:
>>
>> First things first- we are using GNU libc-2.19 that comes packaged in
>> Yocto Poky 'Daisy' release for one of our projects. This is
>> cross-compiled for ARMv7 and we use arm-poky-linux-gnueabi-gcc (GCC)
>> 4.8.2 toolchain.
>>
>> We use the *non-debug* version of libc in our project for obvious size
>> and performance reasons. With the non-debug version, when a user space
>> thread is executing a libc function like poll(), read(), etc., and we
>> attach a remote gdbserver to the process and try to get backtrace, all
>> we see is the following 4 backtrace frames-
>>
>> (gdb) bt
>> #0  0x759abc00 in poll () from /lib/libc.so.6
>> #1  0x759abbf4 in poll () from /lib/libc.so.6
>> #2  0x00182c70 in ?? ()
>> #3  0x00182c70 in ?? ()
>> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
>>
>> This, I can understand, is because the non-debug version is stripped
>> of all debug symbols and hence GDB cannot backtrace. But here is the
>> interesting part- we compiled libc (and all of yocto Poky toolchain
>> components) with ARM unwind table and ARM frame pointers for another
>> reason[1]. And, I know that the version of gdb that we're using
>> (7.6.2) has support to backtrace using ARM unwind tables and frame
>> pointers (see [2] and [3]). But, even then, all we get from GDB
>> backtrace is the above shown output. Does anybody have any comments on
>> why enabling ARM unwind tables and ARM frame pointers in libc still
>> does not give us backtrace? And is there any other way we can get
>> backtrace to go past libc functions without including debug sections
>> in the libc elf?
>
> In fact your debug session GDB is indeed getting a stacktrace, what
> it is not being to do is associate the address with meaningful names.
> This is due not because missing debug information, but because not all
> symbols in the executable are in dynamic symbol table.
>
> Backtrace man pages explains it briefly in 'NOTES' section and you
> need to add -rdynamic compiler flags (which is will be translated to
> -export-dynamic linker option). In a armhf environment using the
> man pages example:
So, I understand that -rdynamic flag will add entries for all symbols
in the dynamic symbol table. But, I think my problem here is different
and not related to symbolicating the addresses. Please allow me to
explain. The addresses without symbols you see in the GDB backtrace
(new one pasted below) are not even in my process' stack regions.

(gdb) bt
#0  0x758b9190 in poll () from /lib/libc.so.6
#1  0x758b9184 in poll () from /lib/libc.so.6
#2  0x013df120 in ?? ()
#3  0x013df120 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

And address 0x013df120 is in the heap region in proc/<pid>/maps (shown
below) of my process-
root@xyz# cat /proc/621/maps
...<snip>...
01389000-0154e000 rw-p 00000000 00:00 0          [heap]
...<snip>...
74941000-74980000 rwxp 00000000 00:00 0          [stack:756]
74980000-74981000 ---p 00000000 00:00 0
74981000-749c0000 rwxp 00000000 00:00 0          [stack:730]
749c0000-749c1000 ---p 00000000 00:00 0
749c1000-74a00000 rwxp 00000000 00:00 0          [stack:729]
74a00000-74a01000 ---p 00000000 00:00 0
74a01000-74a40000 rwxp 00000000 00:00 0          [stack:728]
74a40000-74a41000 ---p 00000000 00:00 0
74a41000-74a80000 rwxp 00000000 00:00 0          [stack:727]
74a80000-74a81000 ---p 00000000 00:00 0
74a81000-74ac0000 rwxp 00000000 00:00 0          [stack:726]
74ac0000-74ac1000 ---p 00000000 00:00 0
74ac1000-74b00000 rwxp 00000000 00:00 0          [stack:725]
74b00000-74b01000 ---p 00000000 00:00 0
74b01000-74b40000 rwxp 00000000 00:00 0          [stack:724]
74b40000-74b41000 ---p 00000000 00:00 0
74b41000-74b80000 rwxp 00000000 00:00 0          [stack:722]
74b80000-74b81000 ---p 00000000 00:00 0
74b81000-74bc0000 rwxp 00000000 00:00 0          [stack:721]
74bc0000-74bc1000 ---p 00000000 00:00 0
74bc1000-74c00000 rwxp 00000000 00:00 0          [stack:720]
74c00000-74c01000 ---p 00000000 00:00 0
74c01000-75400000 rwxp 00000000 00:00 0          [stack:669]
75400000-75500000 rw-p 00000000 00:00 0
75515000-75516000 ---p 00000000 00:00 0
75516000-75555000 rwxp 00000000 00:00 0          [stack:718]
75555000-755c0000 r--p 00000000 00:0b 1131       /usr/share/SetBitmaps.bitmap
755c0000-755c1000 ---p 00000000 00:00 0
755c1000-75600000 rwxp 00000000 00:00 0          [stack:659]
75600000-75621000 rw-p 00000000 00:00 0
75621000-75700000 ---p 00000000 00:00 0
75700000-75701000 ---p 00000000 00:00 0
75701000-75708000 rwxp 00000000 00:00 0          [stack:723]
75708000-75709000 ---p 00000000 00:00 0
75709000-75748000 rwxp 00000000 00:00 0          [stack:658]
75748000-75749000 ---p 00000000 00:00 0
75749000-75788000 rwxp 00000000 00:00 0          [stack:648]
75788000-7578c000 rw-p 00000000 00:00 0
...<snip>...
7592f000-75936000 ---p 0013d000 00:0b 428        /lib/libc-2.19.so
75936000-75938000 r--p 0013c000 00:0b 428        /lib/libc-2.19.so
75938000-75939000 rw-p 0013e000 00:0b 428        /lib/libc-2.19.so
...<snip>...
7e9d8000-7e9f9000 rwxp 00000000 00:00 0          [stack]

So, one of two things could have happened here-
1. GDB backtrace is just unable to get the stacktrace without debug
symbols in libc. Having just ARM unwind tables and ARM frame pointers
is not helping GDB to walk up the stack. This I'm little less
suspicious because I've looked at the code in gdb's ARM port and I
know the code exists there to use unwind tables and frame pointers to
get backtrace.
2. Or (I'm more suspicious about this one)- even though I build glibc
with -fasynchronous-unwind-tables, -fno-omit-frame-pointer and
-mapcs-frame compiler flags, glibc Makefiles may be overriding these
flags for certain files/parts of the libc build and thus preventing
gdb from walking the stack for these functions. Is there an easy way
(I know I can grep and replace in all Makefile and then fix build
errors which is the harder way) to force glibc to use the above 3
flags and not allow the individual Makefiles to override them? And I
want to do all this with a non-debug version of glibc (without .debug
sections in elf).

>
> $ ./test 3
> backtrace() returned 0 addresses
> $ gcc -Wall test.c -O3 -o test -funwind-tables
> $ ./test 3
> backtrace() returned 3 addresses
> ./test() [0x8710]
> ./test() [0x8620]
> /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x99) [0xf76aa632]
> $ gcc -Wall test.c -O3 -o test -funwind-tables -rdynamic
> $ ./test 3
> backtrace() returned 3 addresses
> ./test(myfunc3+0xb) [0x8968]
> ./test(main+0x33) [0x8878]
> /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x99) [0xf70c6632]
>
>
>>
>> Thanks,
>> Sundar Dev
>>
>> [1] In the release version of our code, we don't want to have our
>> applications and libraries built with debug symbols (for reasons of
>> code bloat, performance, etc). But we would still like to get a stack
>> trace in the event of a crash by using the glibc backtrace() and
>> backtrace_symbols_fd() APIs. The libgcc implementation of these APIs
>> works with ARM unwind tables and frame pointers. And hence we build
>> our application and libraries with -fasynchronous-unwind-tables,
>> -fno-omit-frame-pointer and -mapcs-frame flags and it works.
>> [2] https://sourceware.org/ml/gdb-cvs/2011-02/msg00011.html
>> [3] https://sourceware.org/ml/gdb-cvs/2011-02/msg00012.html
>>

More information about the Libc-help mailing list