A public interface for library search

[Kozlukov Sergei]

Hi! I wasn't sure if this was worth a ticket on BugZilla yet, I hope libc-help will be an appropriate venue.

I'm trying to assess if there's such a thing as an agreeable interface for locating the host platform's shared libraries. Specifically, if there's a "conventional" way of predicting, given a soname, what specific file the `dlopen()` would have loaded, but without actually loading it.

This task is, I think, precisely what certain container-related projects try to accomplish by peeking into the /etc/ld.so.cache, with the purpose of locating the userspace drivers that need to be mounted into the container (think mesa or, most realistically, libcuda.so):

- https://github.com/NVIDIA/libnvidia-container/blob/1eb5a30a6ad0415550a9df632ac8832bf7e2bbba/src/nvc_ldcache.c#L370
- https://github.com/NVIDIA/nvidia-container-toolkit/blob/da0755769fee3ab071aaa3b9fac6fe085da7071d/internal/ldcache/ldcache.go#L36

These tools are then used by many downstream projects, e.g. apptainer/sylabs-singularity, docker, etc.

As far as I can tell, /etc/ld.so.cache is glibc internals and isn't meant to be used as a public interface. In particular, NixOS does not populate that file, and the nixpkgs simply patch all software that refers to it.

I think this behaviour has to change upstream, in those projects, and for that we need an interface that could be considered "conventional", "maintstream", and "supported".

The combination of `dlopen` and `RTLD_DI_ORIGIN` are almost perfect for the task: they respect LD_LIBRARY_PATH, LD_PRELOAD, and even the inquiring process's RUNPATH. Unfortunately, that comes with the side-effect of executing the loaded libraries' constructors. I understand that this shouldn't be a security concern, at least in the specific case of nvidia's container-toolkit and from glibc's perspective. However, there's no good reason to actually "load" the libraries being searched for, so I expect that this solution would be, generally, hard to sell.

Thanks!