[Bug dynamic-link/15971] No interface for debugger access to libraries loaded with dlmopen

[eblake at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15971

--- Comment #5 from Eric Blake <eblake at redhat dot com> ---
(In reply to Florian Weimer from comment #4)

Replying out of order:

> This is more of a topic for libc-help. Posting to random bug reports isn't
> really the way to request help from your fellow Red Hatters.

This wasn't a random request for help, so much as confirmation that this
(long-standing) bug is still out there, and that I encountered it because I
came up with a use case where dlmopen() solved a problem for me.  I got my code
to accomplish my goal before finding this bug report, and was merely confirming
that the bug (of gdb not being able to debug dlmopen()d code) is still present,
even if I managed to get my nbdkit patch working in spite of the debugging
deficiency.  If anything, I'm hoping that my post here serves as documentation
for why fixing this bug may have benefits to other users who try dlmopen(),
rather than me needing help.

> (In reply to Eric Blake from comment #3)
> > I ran into this today while working on nbdkit; I needed a way to work around
> > VDDK's buggy proprietary library that calls dlopen("libcrypto.so") on a
> > relative path name but where the library comes with its own version of
> > libcrypto.so that is incompatible with the one in /usr/lib, but I also need
> > a solution that would not require LD_LIBRARY_PATH.
> 
> Why is LD_LIBRARY_PATH not an option?

VDDK's library is proprietary, and installing it puts both libvixDiskLib.so and
a sub-par libstdc++.so (among others) in the same directory. If the user
exports LD_LIBRARY_PATH to point to vddk's library, they break execution of any
binary that depends on a newer libstdc++.so. 
https://bugzilla.redhat.com/show_bug.cgi?id=1756307#c7

At the same time, libvixDiskLib.so exports an initialization function which in
turn calls dlopen("libcrypto.so") and similar to load the versions of libraries
that it shipped with; because VDDK appears to be built without proper rpath,
this load fails if it finds /usr/lib64/libcrypto.so instead of the version it
shipped alongside libvixDiskLib.so.  But because VDDK is proprietary, we can't
rewrite their library to fix their bug.  So the only way to use VDDK is to
influence the search path so that the relative loads performed by VDDK resolve
to VDDK's installation path, but it is desirable to limit this influence to
just the process loading VDDK.

Telling users they have to set LD_LIBRARY_PATH before running nbdkit (where
nbdkit the loads libvixDiskLib.so via dlopen) is annoying: although nbdkit
itself is (so far) not broken by any of the other libraries installed by VDDK
(because it is a C program, not a C++ program), any child process that nbdkit
spawns has to undo the LD_LIBRARY_PATH damage.  Worse, nbdkit is DESIGNED to
spawn a child process, and the main executable that spawns a child process
(nbdkit --run 'command ...') is in a separate binary than the shared library
that dlopen()s libvixDiskLib.so (nbdkit-vddk-plugin.so); coordinating
environment variables between the two binaries introduces awkward coupling
problems.

Since LD_LIBRARY_PATH is unpalatable, I then explored how I could hook into the
dlopen process.  My solution was to dlmopen() a library that hooks dlopen(),
although your suggestion of using la_objsearch() is worth exploring - the fact
that the dlopen man page did not mention la_objsearch, and that the dlmopen man
page did not mention that dlmopen() currently does not support gdb debugging,
could be considered documentation bugs under the umbrella of this bug.

> 
> If the inheritance by subprocesses is a problem, you could try an explicit
> loader invocation with --library-path, or scrub the process environment
> after the process has been loaded. (After program start, changing the
> environment variable does not alter the search path.)

Re-execing nbdkit would involve adding coupling between the main binary and the
dependent library that dlopen's vddk.  An explicit loader invocation implies a
re-exec of nbdkit.

> 
> > My solution was to use
> > dlmopen() to open a shim library defining an alternative dlopen(), so that I
> > could intercept VDDK's poor dlopen calls and replace them with saner
> > absolute loads.
> 
> It looks like you are reimplementing the la_objsearch hook from LD_AUDIT.

The man page for dlopen did not mention la_objsearch.  And, on at least Fedora
31, 'man la_objsearch' fails.  (I did find online references to it, though, now
that you've pointed it out).  Is it something that HAS to go through the
environment variable LD_AUDIT, or can a standalone shared library be its own
auditing interface for just a single process?  If it is really that easy to
hook dlopen() to rewrite relative paths into absolute, it seems like it would
be easier to locate documentation on the matter.

> 
> dlmopen seems the wrong solution for this because many things break with the
> current implementation, not just debugging.

Then those pitfalls should be documented in the dlmopen man page, as well as a
pointer back to this bug (and any others that are the result of difficulties in
process management caused by dlmopen).

-- 
You are receiving this mail because:
You are on the CC list for the bug.