Suggestion: backtrace stop guard for threads callstacks

[Joel Brobecker <brobecker at adacore dot com>]

Hello,

We have recently implemented a small feature in our version of the
debugger. While the implementation itself is not ready for inclusion
here, and not directly interesting in any case (lacking support for
Ada tasks for now, see below), I thought the general idea might be
interesting to the group. And I think it's not going to be too time
consuming, so could probably produce a patch.

One of our customers reported that he saw a warning from the debugger
saying that a frame was identical to the previous frame. Here is the
kind of output he would see:

> #3  0x000370d4 in system.tasking.stages.task_wrapper ()
> #4  0xff34b124 in _thread_start () from /usr/lib/libthread.so.1
> #5  0xff34b124 in _thread_start () from /usr/lib/libthread.so.1
> warning: Previous frame identical to this frame (corrupt stack?)

In this case, the user tried to print a backtrace from inside an Ada
task, which is mapped over a thread, and that the unwinder didn't
(couldn't?) know that it was time to stop at frame #4. The same sort
of situation happens with the main process, outside of a thread context,
where we sometimes cannot know that we reached the top of the callstack.

For the main process, we have a guard put in place that will stop it
when reaching the "main program". See main_name(), inside_main_func(),
and get_prev_frame().

What we've done for Ada tasks, is to take advantage of the fact that
we know that tasks are run from a known routine in the GNAT runtime,
called System.Tasking.[...].Task_Wrapper. So we added a small piece
of code to detect Ada frames corresponding to this routine (we actually
have two, but the principle is the same), and declare that we're
inside the main func if the task_wrapper is detected.

The complete backtrace can be obtained using the already implemented

        (gdb) set backtrace past-main

It's a little bit less simple for the general case for C, C++ and other
languages, but I suggest we implement something similar. How about we
add some code that recognizes some of the thread entry-points? For
the GNU/Linux NPTL for instance, we could recognize clone() or perhaps
_start_thread? We could make the check as stringent as we like, for
instance not only check the funtion name, but also the object name,
to make sure we're inside the pthread library.

Just throwing some ideas.

My overall idea is to implement a mechanism where various modules
in GDB could register their own routine that would be able to tell
whether they recognize the given function as a main or not.

For instance, sol-thread.c could implement its routine, and register
it during the init phase... inside_main_func() would then go through
this list, and call each one of them until one recognizes it or until
it reaches its end.

Even grander, have a language-specific-hook, followed by list of
registered sniffers. So inside_main_func() would look like this:

    msymbol = lookup_minimal_symbol (main_name (), NULL, symfile_objfile);
    [check this symbol to see if it's the main]

    /* Check this frame using the language-specific sniffer */
    symtab = find_pc_symtab (pc);
    if (symtab != NULL)
      if (language_main_name_sniffer[language] (this_frame))
        return 1;

    /* Check using the other sniffers  */
    FOR_ALL_MAIN_FUNC_SNIFFERS (sniffer)
      if (sniffer (this_frame))
        return 1;

    return 0;

Thoughts?
-- 
Joel