dlclose attempts to unload/unmap the DSO when closing the last reference. Unfortunately, this operation is fundamentally unsafe. Consider a program "main" that links to a library "foo" and dynamically loads another library "bar" which depends on "foo". In this situation, "bar" is a candidate for unloading, but "foo" is not. Suppose "bar" was not designed specifically for use as a dynamic loaded module, just an ordinary library, and suppose "bar" leaks a reference to functions or objects in its mapping down to "foo" -- for example, by registering a widget/codec/converter/etc. of some sort for use by "foo". After "main" closes "bar", the leaked reference remains in storage belonging to "foo", and will result in UB (crash or worse) when "foo" later attempts to dereference it.
Note that the problem would not arise if both "foo" and "bar" had been loaded dynamically with the same reference count (i.e. with "bar" as the only user of "foo") since "foo" would also get unloaded at this point; nor would it arise if both were linked into "main".
I'm attaching minimal test cases to demonstrate the problem. The "registration with foo" concept is idiotically oversimplified in the test case, but you can imagine it being some fancier data structure that allows multiple registration and perhaps even unregistration.
Moreover, this kind of issue is not the ONLY way dlclose can break things; it's just one example. Another example would be a library which starts a thread the first time it's called and never reports the fact that it started a thread to the calling application. This will of course crash immediately when dlclose is called.
For better or worse, there is no way to fix the problem outright without disabling unmapping on dlclose entirely. As a fix, I recommend disabling unmapping, and creating a new DT_GNU_UNLOADABLE tag or other similar mechanism that DSOs can use to tag themselves as safe for unloading. Then, DSOs being built with the intent of being plugins/loadable-modules can be written to be unload-safe and tagged as such, and the dynamic linker will no longer trash the process when closing a plain library file that was not intended to be unloadable.
Created attachment 6601 [details]
source for libfoo.so
compile with gcc -shared -o libfoo.so foo.c
Created attachment 6602 [details]
source to libbar.so
compile with gcc -shared -o libbar.so bar.c -L. -lfoo
Created attachment 6603 [details]
source for main test program
compile with gcc main.c -L. -lfoo -ldl to see the bug; add -lbar to see it go away
> Unfortunately, this operation is fundamentally unsafe.
No, it isn't. Unloading a library that wasn't designed to be unloadable is unsafe, so don't do that.
Unloading in general is very useful, so blank disable of unloading you proposed would be akin to throwing the baby out with the bathwater.
Finally, a library can be linked with -Wl,-z,nodelete, which would prevent it from ever being unloaded.
The testcase is invalid. As Paul said, you can use DF_1_NODELETE on libraries that you don't want to unload ever, or if a library (libfoo) in this case performs a cal to a function from libbar, it will have a dynamic dependency on libbar through symbol resolution and thus would allow libbar to be unmapped only when libfoo is about to go away too.
Add an extra level of libraries then. For example, consider myplugin.so that depends on libbar.so which depends on libfoo.so. If myplugin.so is designed to be unload-safe, but libbar.so isn't, the application has no way of knowing it's unsafe to call dlclose on myplugin.so.
I agree DF_1_NODELETE/-Wl,-z,nodelete is part of the fix, but the default is backwards. A library is not safely unloadable by default. It's only safely unloadable if it was explicitly designed to be. Perhaps if libtool added -Wl,-z,nodelete by default except when configured not to, that would solve the problem...
Doesn't matter what levels of libraries you have. It is users responsibility to ensure pointers to library code aren't accessed after library is unloaded. The default is correct, and has been that way since the beginning.