[Libguestfs] alternatives for hooking dlopen() without LD_LIBRARY_PATH or LD_AUDIT?

[Eric Blake]

On 2/17/20 9:12 AM, Florian Weimer wrote:
> * Eric Blake:
> 
>> I'm just now noticing that 'man ld' reports that you may pass '--audit
>> LIB' during linking to add a DT_DEPAUDIT dependency on a library
>> implementing the audit interface, which sounds like it might be an
>> alternative to LD_AUDIT for getting a library with la_objsearch() to
>> actually do something (but doesn't obviate the need for la_obsearch()
>> to be in a separate library, rather than part of the main executable,
>> unless a library can be reused as its own audit library...).
> 
> DT_AUDIT support has yet to be implemented in glibc:
> 
>    <https://sourceware.org/bugzilla/show_bug.cgi?id=24943>
>    <https://sourceware.org/ml/libc-alpha/2019-08/msg00705.html>
> 
> If you go on record saying that you need this, maybe someone will review
> the patch.  Sorry. 8-(

Another followup: nbdkit-vddk-plugin.so is now using a re-exec setup 
[1], for several reasons:
1. all our other ideas tried so far (such as a dlopen() interposition in 
the main nbdkit binary) touched more files and required more exported 
APIs; we managed to get re-exec working with changes limited to just the 
plugin (other than a minor change to nbdkit to quit messing with argv[] 
so that /proc/self/cmdline would stay stable - but that did not require 
a new API).
2. it turns out that overriding dlopen() was insufficient to work around 
VDDK's setup [2].  It _did_ solve the initial dlopen() performed by 
VDDK, but that library in turn had DT_NEEDED entries for bare library 
names, which dlopen() does NOT affect but which la_objsearch() should.
3. for nbdkit, we want to minimize the number of binary files shipped; 
the re-exec solution works with just the nbdkit binary and the 
nbdkit-vddk-plugin.so.  Any solution that requires a third file to be 
shipped (be that a shared library providing dlopen, or a LD_AUDIT 
library, or otherwise) is less palatable than the 2-binary solution that 
our re-exec solution provides.

[1] https://github.com/libguestfs/nbdkit/commit/0c7ac4e655b
[2] https://www.redhat.com/archives/libguestfs/2020-February/msg00184.html

So with that said, here's a question I just thought of:

If your patch for glibc support for DT_AUDIT is incorporated, is it 
possible to mark a shared library as its own audit library via DT_AUDIT? 
  That is, if nbdkit-vddk-plugin.so can provide entry points for _both_ 
the nbdkit interface (which satisfies dlopen() from the nbdkit binary) 
and la_version/la_objsearch() (which satisfy the requirements for use 
from the audit code in ld.so), _and_ during the compilation of 
nbdkit-vddk-plugin.so, we marked the library as its own DT_AUDIT entry, 
would the mere act of dlopen("nbdkit-vddk-plugin.so") from nbdkit be 
sufficient to trigger audit actions such as la_objsearch() on all 
subsequent shared loads (whether by dlopen() or DT_NEEDED) performed by 
nbdkit-vddk-plugin.so and its descendant loaded libraries?  Because if 
so, we would have a use case where a single binary, set up to act as its 
own audit library, might be sufficient to hook the shared object search 
path without needing any of environment variable modification, a process 
re-exec, or a third shipping binary - in which case that would indeed be 
a nicer solution than the current re-exec solution we committed today 
(of course, nbdkit would not be able to rely on that solution except on 
systems with new enough glibc to support DT_AUDIT).

I guess even without DT_AUDIT support, I could at least answer the 
question of whether a single .so can be used to satisfy both dlopen() 
and LD_AUDIT interfaces at once by setting LD_AUDIT (where the only 
remaining gap is figuring out when glibc can let DT_AUDIT have the same 
effect).  During my earlier attempts to get a working dlopen() override, 
I didn't consider any solution that required setting LD_AUDIT, but now 
that I proved dlopen() override alone was not enough for the case at 
hand, having to re-exec to set LD_AUDIT is no worse than having to 
re-exec to set LD_LIBRARY_PATH as a fallback to systems where glibc does 
not support DT_AUDIT.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org