[i386/fyi] small adjustment to i386 frame code

[Joel Brobecker]

Hello,

Since this has been discussed here in several occasions...

Since GDB 6.0, the debugger has been using a more sophisticated
frame unwinder. It used to blindly follow the %ebp register, but
no longer does that unless it is sure that the frame base address
has been saved there.

Although the new frame code gives more much more accurate results
most of the time, there are some occasional cases where we have
to unwind through functions which do not follow the ABI, and for
which no frame debugging info seem to be provided. In these particular
cases, the unwinder fails, and a backtrace from such function aborts
prematurely. One such example is WaitForSingleObjectEx (in one of
the windows DLL). I posted the assembly of this function a long while
ago, it's just plain impossible to determine where the frame base is.

Unfortunately for us, that function is used fairly regularly by any
application which would use tasking (a high-level equivalent of threads
in C), which is something relatively common in the Ada world. So we
really need to be able to unwind past that function.

When it comes to unwinding using prologue analysis, there is probably
no perfect solution. Here is the approach we are going to try out:
Follow blindly the %ebp register for "the complicated functions".
And because we've only encountered this type of code in a DLL, we've
limited this hack to code in DLLs. I added a comment in the code
explaining the condition that implements "complicated".

We know there is a risk of %ebp being clobbered, we also know that
%ebp is the base address of the caller's frame, now ours. But that
will at least give us a chance of having the backtrace reach the user
code.

If this is something of general interest, in this form or any variation,
I would be happy to contribute it.

-- 
Joel
-------------- next part --------------
Index: i386-tdep.c
===================================================================
RCS file: /gnat.dev/cvs/Dev/gdb/gdb-6.4/gdb/i386-tdep.c,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -p -r1.4 -r1.5
--- i386-tdep.c	10 Aug 2006 22:18:30 -0000	1.4
+++ i386-tdep.c	22 Aug 2006 21:45:53 -0000	1.5
@@ -1029,6 +1029,41 @@ i386_frameless_adjust_cache_hack (struct
     }
 }
 
+/* Return non-zero if the function starting at START_PC has a prologue
+   that sets up a standard frame.  */
+
+static int
+i386_function_has_frame (CORE_ADDR start_pc)
+{
+  struct i386_frame_cache cache;
+
+  cache.locals = -1;
+  i386_analyze_prologue (start_pc, 0xffffffff, &cache);
+
+  return (cache.locals >= 0);
+}
+
+/* Return non-zero if PC is inside one of the inferior's DLLs.  */
+
+static int
+i386_in_dll (CORE_ADDR pc)
+{
+   char *so_name = solib_address (pc);
+   int len;
+
+   if (so_name == NULL)
+     return 0;
+
+   len = strlen (so_name);
+   if (len < 5)
+     return 0;
+
+   return ((so_name[len - 1] == 'l' || so_name[len - 1] == 'L')
+           && (so_name[len - 2] == 'l' || so_name[len - 2] == 'L')
+           && (so_name[len - 3] == 'd' || so_name[len - 3] == 'D')
+           && so_name[len - 4] == '.');
+}
+
 /* Normal frames.  */
 
 static struct i386_frame_cache *
@@ -1075,10 +1110,38 @@ i386_frame_cache (struct frame_info *nex
 	 frame by looking at the stack pointer.  For truly "frameless"
 	 functions this might work too.  */
 
-      i386_frameless_adjust_cache_hack (cache, frame_pc_unwind (next_frame));
+      if (i386_in_dll (cache->pc)
+          && !i386_function_has_frame (cache->pc))
+        {
+          /* Functions in DLL for which do not seem to create a standard
+             frame are unwound using %ebp.  This is actually the caller's
+             frame base instead of our own, but there are some functions
+             such as WaitForSingleObjectEx in one of the Windows system
+             DLLs for which the frame base cannot possibly be determined
+             from the stack pointer.  As a consequence, our caller will be
+             missing from the backtrace, but this is better than having
+             an aborted backtrace due to a bogus frame base.
+             
+             We use this approach only for functions in DLLs because
+             this is the only place where we have seen the type of
+             highly optimized code that cause us trouble.  In other
+             cases, we expect the code to come with frame debugging
+             information, making prologue scanning unnecessary.
+             
+             We also avoid blindly following %ebp if we are midway through
+             setting up a standard frame.  In that case, we know how to
+             determine the frame base using the stack pointer.  */
 
-      frame_unwind_register (next_frame, I386_ESP_REGNUM, buf);
-      cache->base = extract_unsigned_integer (buf, 4) + cache->sp_offset;
+          cache->saved_regs[I386_EBP_REGNUM] = 0;
+        }
+      else
+        {
+          i386_frameless_adjust_cache_hack (cache,
+                                            frame_pc_unwind (next_frame));
+ 
+          frame_unwind_register (next_frame, I386_ESP_REGNUM, buf);
+          cache->base = extract_unsigned_integer (buf, 4) + cache->sp_offset;
+        }
     }
 
   /* Now that we have the base address for the stack frame we can