Dynamic library loading inner workings

Andreas Fink andreas.fink85@googlemail.com
Sun Dec 5 18:44:00 GMT 2010


On Sun, 5 Dec 2010 11:34:28 -0500
"Carlos O'Donell" <carlos@systemhalted.org> wrote:

> On Thu, Dec 2, 2010 at 5:13 PM, Andreas Fink
> <andreas.fink85@googlemail.com> wrote:
> > Thanks for the relocation keyword. I tried what you've suggested and I was running it with LD_DEBUG=all. I think this is the line which is interesting:
> > file=/usr/lib/qt4/libQtGui.so [0];  needed by /usr/lib64/qt4/libQtCore.so.4 [0] (relocation dependency)
> >
> > Several questions come to my mind:
> > - Do I understand it correctly, that libQtCore.so needs libQtGui.so, so this is the reason why unloading libQtGui will fail?
> 
> Yes.
> 
> > - What is a relocation dependency (maybe you have some informative link, couldn't find anything useful with google except the dl-lookup.c)?
> 
> You need to learn about relocations in general to understand the
> context, for any beginner I strongly recommend reading "Linkers &
> Loaders" by Levine
> (http://www.goodreads.com/book/show/1103509.Linkers_and_Loaders).
> 
> A relocation dependency occurs when an address in one shared library
> (libQtCore.so) is relocated to point at data or code in another shared
> library (libQtGUI.so). Unloading the library that is dependend upon
> would leave the dependent library with an invalid reference.
> 
> In general it is this type of complexity that created the loose
> requirements around dlclose() needing to do anything concrete (that
> and performance reasons).
> 
> > - Why can't there be a simple reference counting for dependent libraries (libQtGui needs libQtCore --> increase reference counter, libQtXml needs libQtCore --> increase reference counter, etc)
> 
> There is a simple reference count. That's how it's implemented.
> 
> > - Is there any way to make this work in a generic manner (in this case preloading libQtCore would solve it, but what if I have different libraries with the same behaviour)?
> 
> There is no generic way to handle this issue, and there are no
> assurances that dlclose() will unload the shared library.
> 
> Why do you want more assurances?
> 
> What problem are you trying to solve?
> 
> Cheers,
> Carlos.

Hello,

I've been reading a bit the last days about this topic (really interesting, as I must admit).

My problem looks like this:
Abstractly seen I have two types of libraries. Libraries of type A are called libA, libB, libC. Libraries of type B are called libX, libY, libZ. Libraries of type A are quite stable and good to ship. Libraries of type B are very unstable and in development. However libraries of type B link against libraries of type A, and libraries of type B are implemented as plugins (so they are loaded via dlopen in my program).
libX links against libA, libB
libY links against libB, libC
libZ links against libA, libC

Loading all three of them (in the order libX, libY and then libZ) makes it impossible to only unload libX (since libA and libB have a dependency on libX because of relocation). So I would need to unload all libX, libY, and libZ to be able to reload a recompiled libX.


Right now, I'm preloading libA, libB and libC, which works perfectly fine. However other people are going to use my program too, so this does not seem to be a nice solution (and other people are writing their own plugins, so they probably need to preload other libraries).

Anyway it does not seem that my problem can be solved with the current linker. I have another question to the order libraries are loaded.
If I dlopen(libX) and look at the order in which symbols are tried to be resolved I see that symbols in libX are first looked at before libA and libB. Thus when relocation of libA happens a symbol is found in libX (which would also exist in libA, but the one in libX is used and hence the relocation dependency of libA to libX occurs).

So my new question:
Why is the linker not putting dependent libraries before the loaded library, i.e. the current situation looks like this:
dlopen(libX) --> symbol resolution in this order:
(... some libraries loaded at program startup)
libX
libA
libB

my preferred solution:
(... some libraries loaded at startup)
libA
libB
libX

Then the relocation dependency of libA on libX would not occur since all symbols in libA can be resolved with symbols in libraries prior to libA or libA itself.

To summarize my whole point:
I understand now, why libQtGui cannot be unloaded, but I don't understand why symbol resolution happens in a (for me) quite unnatural order... My natural order for symbol resolution on dlopen(libX) would be: Libraries loaded at program startup --> dependent libraries of libX (libA, libB) --> libX itself

Hopefully my point is clear.
Regards
Andreas



More information about the Libc-help mailing list