ELF shared library breakpoints

[Mark Kettenis]

   Date: Fri, 4 Jun 1999 11:35:03 -0700
   From: Stan Shebs <shebs@cygnus.com>

      If calling disable_breakpoints_in_shlibs only breaks SunOS 4, which I
      suspect since there is also a bug report concerning Solaris, the code
      could be wrapped in #ifdef SVR4_SHARED_LIBS.

   That seems rational, although I want to understand the issue this
   time instead of fixing symptoms.  Then we can write a comment on this
   and not have to be puzzled ever again (on this subject anyway :-) ).

Always a good idea :-).  Let me try to help you.  However, be aware
that I have no experience with SunOS 4.  I know the GNU dynamic linker
(used by Linux and the Hurd) fairly well.  And since the GNU dynamic
linker is modelled after the Solaris 2 dynamic linker, my knowledge
will probably apply to Solaris 2 too.

Now let's assume we have been debugging a dynamically linked program,
that we have placed some breakpoints in one of the shared libraries.
Gdb keeps track of these breakpoints by their memory location.  Now we
kill the programs, and restart it.  The dynamic linker and the program
itself will be mapped into memory, but the shared libraries won't.  It
is the task of the dynamic linker to do that before running the
program itself.  When the dynamic linker is about to be executed, a
SIGTRAP is sent to gdb.  When gdb receives this signal, one if the
things it does is reinserting the breakpoints into th program.  On the
i386 this means inserting a trap instruction at the memory location of
the breakpoint.  This is no problem for breakpoints in the program
itself, but since the shared libraries have not yet been mapped into
memory, accessing the memory location at which the breakpoints have
been placed will fail, and gdb refuses to run the program.

To circumvent this problem, the idea is to temporarily disable
breakpoints in shared libraries when restarting a program, such that
they will be ignored when gdb tries to reinsert its breakpoints.
Later, when all shared libraries have been mapped into memory, gdb
goes over all breakpoints again, reinserting the temporarily disabled
shared library breakpoints.

Temporarily disabling breakpoints is exactly what
`disable_breakpoints_in_shlibs' does, and since it has to be done
before restarting the pogram, `clear_solib' seems to be the right
place to do it.

The implementation of shread libraries in SunOS 4 significantly
differs from ELF shared libraries (that's what SVR4_SHARED_LIBS are),
and I don't really know how they work.  It is possible that on SunOS
4, the shared libraries are mapped into memory together with the
dynamically linked program itself, which would make temporariliy
disabling the shared library breakpoints unecesary.  If we then
disable the shared library breakpoints anyway, they will probably
never be re-enabled again which will indead break rerunning.

I suspect that the only reason gdb support for SunOS 4 shared libraries
and ELF shared libraries live together in one file, is to support
debugging SunOS 4 programs on Solaris 2.

Mark