Loading glibc in a new namespace fails vtable check

[Moshe Rubin]

@Florian Kudos for your prompt reply.  After a long debugging session
yesterday I was able to get to the core of the issue, and you were correct:
the backtrace was not directly due to loading a second glibc in a second
namespace.  I will briefly explain why the fatal error occurred.

My two apps, both the POC and the production, both used dlmopen("glibc.so")
to load a second glibc instance in a second namespace.  They differed,
however, in what they did following dlmopen() and dlsym(handle, "...").

In the production app, once dlmopen() completed, the code called
dlsym(handle) to retrieve function pointers to fopen, fread, fwrite,
fclose, fileno, and other functions (we'll call these function pointers
p_fopen, p_fread, p_fwrite, etc.).  Using an LD-version script (
https://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_25.html) my
production app intercepts all calls to these functions, calling the
corresponding p_* function residing in the second namespace.

Thus, when any module makes a call to fopen(), it is intercepted by my
code, which calls the identical function but in the second namespace (i.e.,
f_fopen)..

The fatal error was caused when the above functions were passed a FILE* of
stdin/stdout/stderr.  Calling p_fwrite(stdout) is essentially calling the
fwrite() function in the second namespace with a FILE* created in the
default namespace.  This is precisely what the _IO_vtable_check() function
checks for.  When this occurred, the fatal error was triggered.

There are other technical issues that complicate the solution, but at least
I understand what caused the fatal error.

Moshe


On Wed, Sep 25, 2024 at 10:50 AM Florian Weimer <fweimer@redhat.com> wrote:

> * Moshe Rubin via Libc-help:
>
> > Question: Can I assume this trace shows the second glibc copy being
> loaded
> > into the new namespace?
>
> I don't see evidence that a second libc is involved at this point.
> Unless you have a very recent toolchain version, if GDB shows a full
> backtrace, no dlmopen has happened.  I'm not even sure that all the
> necessary changes have actually been merged into GDB.  So that's another
> indicator that there is no dlmopen.  Setting a breakpoint on dlmopen
> would allow you to confirm whether it gets called (that works because
> there are no multiple namespaces when the breakpoint is hit).
> LD_DEBUG=all help could help as well.
>
> I would look for some form of memory corruption, unrelated to future use
> of dlmopen.
>
> Thanks,
> Florian
>
>