Re: shared library loading performance and prelinking

[Robert Schweikert <rjschwei at abaqus dot com>]

Andreas,

When I set LD_DEBUG on SuSE 7.3, which I tried before I only get the
following information,

24194:	                 number of relocations: 46819
24194:	      number of relocations from cache: 68032

and

24205:	                 number of relocations: 47853
24205:	      number of relocations from cache: 81387

our app has 2 processes, no time information is printed. I already tried
to change the way we bind, NOW vs. LAZY by changing the way we call
dlopen but the change in performance between NOW and LAZY was very
small.

Another interesting thing I noticed is that the number of relocations is
only printed when the process starts, i.e. the initial set of shared
objects is loaded. Loading subsequent shared objects does not produce
debug print. This means that for more than half of the shared objects I
cannot get information on the number of relocations.

Also you mentioned that SuSE 8.1 has glibc 2.3.2, did you mean the
upcomming SuSE 8.2 release? We just installed SuSE 8.1 on a machine and
glibc was still 2.2.5.

-> cat /etc/SuSE-release
SuSE Linux 8.1 (i386)
VERSION = 8.1

-> rpm -qa | grep glibc
glibc-2.2.5-152

-> libc.so.6 --version 
GNU C Library stable release version 2.2.5, by Roland McGrath et al.

Did we miss something during install?

Thanks,
Robert

On Sun, 2003-03-16 at 02:19, Andreas Jaeger wrote:
> Robert Schweikert <rjschwei at cox dot net> writes:
> 
> > Hi all,
> >
> > I have a question about library load performance. We have a large
> > application that has 125 shared libraries and a very large number of
> > symbols, lots of them from C++ code. One of the issue we are having is
> > application start up. Not all libraries are loaded at start up. Most
> > libraries are loaded while the user interacts with the program.
> >
> > Since loading is an issue I have been trying to learn about the issues
> > that go along with shared object loading on Linux. One thing that we are
> > trying is to use export lists as suggested in Ulrich's paper. We are
> > still having some problems with it but that's a topic for another post.
> >
> > In my search for answers I have run across prelinking and I was trying
> > to figure out how it works and what it will do for me as far as load
> > performance is concerned. Everything I found said the feature is
> > available only in glivc 2.3.x, thus I am waiting for RedHat 8.1 since
> > earlier distributions still have glibc 2.2.x. Anyway, can someone give
> > me an idea what is happening when prelinking is used, what kind of
> > performance gains one can expect and how to use the prelinking feature?
> 
> First thing is to measure how long relocations at startup use at all.
> glibc comes with LD_DEBUG for this, e.g. try (this is a glibc 2.2.5):
> $ LD_DEBUG=statistics /bin/ls  df 
> 12832:
> 12832:  runtime linker statistics:
> 12832:    total startup time in dynamic loader: 964121 clock cycles
> 12832:              time needed for relocation: 661825 clock cycles (68.6%)
> 12832:                   number of relocations: 275
> 12832:        number of relocations from cache: 101
> 12832:             time needed to load objects: 207215 clock cycles (21.4%)
> /bin/ls: df: No such file or directory
> 
> Other stuff to try is LD_BIND_NOW, it increases the startup time since
> all relocations will be done at start-up time but it should give you
> some statistics:
> $ LD_DEBUG=statistics LD_BIND_NOW=1 /bin/ls  df
> 13612:
> 13612:  runtime linker statistics:
> 13612:    total startup time in dynamic loader: 1788165 clock cycles
> 13612:              time needed for relocation: 1484874 clock cycles (83.0%)
> 13612:                   number of relocations: 1016
> 13612:        number of relocations from cache: 101
> 13612:             time needed to load objects: 207482 clock cycles (11.6%)
> /bin/ls: df: No such file or directory
> 
> Now test this with your application - now you know how much time is
> spend in the relocations and how much prelink could bring at all
> (bringing down that time by around 90 % according to my tests).
> 
> I don't have timings for this but to see the effect of prelinking have
> a look:
> # LD_DEBUG=statistics ls /tmp/q
>      27689:                      number of relocations: 125
>      27689:           number of relocations from cache: 8
> ls: /tmp/q: No such file or directory
>      27689:
>      27689:     runtime linker statistics:
>      27689:                final number of relocations: 187
>      27689:     final number of relocations from cache: 8
> # prelink /bin/ls
> # LD_DEBUG=statistics ls /tmp/q
>      27704:                      number of relocations: 0
>      27704:           number of relocations from cache: 46
> ls: /tmp/q: No such file or directory
>      27704:
>      27704:     runtime linker statistics:
>      27704:                final number of relocations: 0
>      27704:     final number of relocations from cache: 46
> 
> prelink comes with documentation, just read the manual page or run
> prelink --help
> to get started.
> 
> Btw. Debian has switched to glibc 2.3.1 and SuSE 8.1 uses 2.3.2 with
> prelink support.
> 
> Andreas