The gdb x86 function prologue parser

[Jason Molenda]

Hi Mark,

On Jun 12, 2005, at 12:07 AM, Mark Kettenis wrote:

> Sorry I didn't reply before; I still was on vacation when you sent out
> this mail.

Not at all.  Thanks for taking the time to look at it.

> Another question that I have is what calling convention MacOS X will
> use.  Is it something like the System V ABI where the caller is
> supposed to clean up the stack after a function call?

It's the System V IA-32 ABI except that small structs are returned in  
registers, stack frames are 16-byte aligned, and large types are kept  
at their natural alignment.  We haven't officially finalized our x86  
ABI yet, so it's possible we'll tweak it a bit before we're done.   
But it's a safe bet that any changes will be minor variations from  
the SysV IA32 ABI.

> I don't know to what extent your version of gdb is synched with the
> FSF tree, but if it is anything close to gdb 6.x, then yes, you're
> pretty much hosed if the prologue scanner fails.

Yeah, our currently-shipping sources are from a merge circa March  
2004.  We're starting to work on another merge to the current FSF tree.


> It's no surprise to
> me though that the prologue scanner appears a bit weak to you though.
> It started out as a prologue scanner for the origional System V
> compiler with some additions to older GCC versions when I took over.
> At that point, GCC still had a fixed prologue.  When GCC 3.x started
> scheduling prologue instructions, it also started generating usable
> DWARF CFI, so whe took the conscious decision to rely on that and only
> improve the prologue scanner on an as-needed basis.  Since GCC 3.x
> targets for all major free OS'es all use DWARF by default, this means
> that the prologue scanner really only handles some bits of
> hand-optimized assembler.


Ah!  Now it starts to make sense.  I couldn't understand how this had  
been so untested. :)

The one part I'm curious about -- does gdb get the CFI information  
out of gcc's eh_frame section or something?  How do developers debug  
KDE/GNOME applications, where many functions on their stack are from  
optimized libraries that don't have any debug info (except maybe  
eh_frame)?  It seems like these users should be tripping on these  
problems all the time.

>    4. new function, i386_find_esp_adjustments().  This is used in a
>
>
> Being able to debug -fomit-frame-pointer code without CFI probably

(I was only handling -fomit-leaf-frame-pointer)

> means that instead of scanning the prologue, we'll have to scan the
> complete function up to the current instruction pointer.  I really
> wonder if that's the way we should go.

Yeah, my method is not failsafe.  I was trying to handle the case  
where the prologue adjusts the stack pointer a constant amount for  
the lifetime of the function.  If a function has something like an  
alloca() call in it with a non-constant size, I'm going to lose.  If  
I had to scan through the entire function, I'm likely to hit  
instructions I don't understand and stop -- stack adjustments past  
that point would not work.  The only real way to do this is with CFI  
and changes like those in this proposal:

http://dwarf.freestandards.org/ShowIssue.php?issue=030812.2

(which, incidentally, is up for committee review tomorrow morning --  
there's some more detailed proposed text kicking around.)  I don't  
think gcc emits anything useful to help gdb in a case like alloca 
(argc) today, but should be possible to express.

> Well, it handles most of the frameless functions encountered on a
> GNU/Linux system with GCC 3.2 fine.

I tested the patch on a FedoraCore 2 system -- I have no idea which  
compiler was used to build the system, but I'd guess it's gcc 3.2 or  
3.3  A common idiom in this release of gcc is

0xa3b990 <htmlDefaultSAXHandlerInit>:   call   0xa3ba6d  
<htmlDefaultSAXHandlerInit+221>
0xa3b995 <htmlDefaultSAXHandlerInit+5>: add    $0x628df,%ecx
0xa3b99b <htmlDefaultSAXHandlerInit+11>:        push   %ebp
0xa3b99c <htmlDefaultSAXHandlerInit+12>:        mov    0xffffff5c(% 
ecx),%eax
0xa3b9a2 <htmlDefaultSAXHandlerInit+18>:        mov    0xffffff08(% 
ecx),%edx
0xa3b9a8 <htmlDefaultSAXHandlerInit+24>:        mov    %esp,%ebp
0xa3b9aa <htmlDefaultSAXHandlerInit+26>:        movl   $0x0,(%eax)
0xa3b9b0 <htmlDefaultSAXHandlerInit+32>:        mov    %edx,0x14(%eax)
0xa3b9b3 <htmlDefaultSAXHandlerInit+35>:        mov    0xfffffee0(% 
ecx),%edx
0xa3b9b9 <htmlDefaultSAXHandlerInit+41>:        movl   $0x0,0x4(%eax)

and

0xa3ba6d <htmlDefaultSAXHandlerInit+221>:       mov    (%esp),%ecx
0xa3ba70 <htmlDefaultSAXHandlerInit+224>:       ret

It's the standard sequence to find the current PC so static data can  
be located.  The add imm32, r32 isn't recognized by the prologue  
parser, so if you have a C file like

main ()
{
   foo ();
}
foo ()
{
   htmlDefaultSAXHandlerInit();  // invalid, but I'm not going to let  
it run
}

put a breakpoint on main, run the program and put a breakpoint after  
the mov %esp,%ebp:

(gdb) b *0xa3b9aa
Breakpoint 2 at 0xa3b9aa
(gdb) c
Continuing.

Breakpoint 2, 0x00a3b9aa in htmlDefaultSAXHandlerInit ()
    from /usr/lib/libxml.so.1
(gdb) bt
#0  0x00a3b9aa in htmlDefaultSAXHandlerInit () from /usr/lib/libxml.so.1
#1  0xfef1ecf8 in ?? ()
#2  0x080485c3 in foo () at /tmp/a.c:10
Previous frame identical to this frame (corrupt stack?)
(gdb)


It might seem like I'm being mean and picking the one routine that  
fails, but no, this idiom is used in many places and gcc often  
schedules it right into the prologue.  It's a system library, so  
there's no debug info to help you out.

I honestly think that gdb users are hitting this a lot more often  
than you'd think, but aren't reporting it.


> And no, assuming that a function
> has set up a stack frame isn't right; it makes gdb silently skip
> function calls in backtraces.  That can be very confusing.

On 0th frame, you're right, it's wrong to assume that the function  
sets up a stack frame.  But when you're off the 0th frame (and the  
next_frame isn't a signal trampoline or gdb dummy frame, etc.), the  
function set up a stack frame (assuming -fomit-frame-pointer codegen  
wasn't used, in which case we're in real trouble ;).  If we fail to  
parse the prologue instructions in that function, we should assume  
what must be the case - that it set up a stack frame.


> Assuming that a function saves the previous value of %ebp is demanded
> by the System V ABI, but GCC might violate the ABI for static
> functions where it knows the caller has already saved %ebp.

Yeah, rth was telling me the other week that gcc 4 can even ignore  
the ABI for such functions and pass arguments in registers.  My  
changes aren't the only problem in such a scenario.

> Great.  I haven't looked at your patch in detail yet.  But it sounds
> like some of the improvements can be made right away, so let's get
> working on this ;-).


Yeah, I need to integrate the feedback I received last week from  
Daniel and Eli -- lose inttypes.h, use the opcodes instruction parser  
instead of doing it myself -- so I'll get that stuff wrapped up and  
post an updated patch soon.

Thanks again for all the helpful notes,

Jason