Performance problem of dlopen

[Joerg Budischewski]

Hi Jakub,

 >>In numbers, on my system the library takes 3300ms to load with one
 >>dlopen, while using multiple dlopen takes 2100ms. I have actually
 >>played
 >>a little around with LD_DEBUG=symbols and found out, that in the first
 >>case, lookup for symbols which are located in dependent libraries are
 >>searched also in the higher libraries, which leads to significantly
 >>more
 >>lookups.
 >>
 >>So my question is if this is a desired behaviour or can this be
 >>considered as a performance bug ?
 >
 >
 > I'd prefer to see your testproglets first. ld.so has to obey ELF
 > lookup rules, so it is questionable whether anything can be done here.
ok, I have a attached a small C program which just loads shared 
libraries given on command line and prints out the needed time. You can 
simply build it with

gcc -ldl -o shlload shlload.c
.

Additionally I have attached a textfile with a list of libraries for the 
643 (developer) build of openoffice (note: you may also use a production 
build 1.0.x, simply replace 643 by 641. Some libraries may have changed, 
but the effect is still measureable).

Make sure to have a . in your LD_LIBRARY_PATH. Starting

./shlload -v libwrp643li.so

Gives on my system 3150ms needed to load.

Using
xargs ./shlload -v < lst

gives about 1850ms on my system.

 > What would IMHO help most for OpenOffice startup spent in the dynamic
 > linker
 > (in addition to aggressive profiling and speeding up often used
 > functions)
 > is to merge shared libraries which are
 > a) linked statically to the OOo programs
 > b) dlopened during start of every OOo program (or at least all the
 > important
 >    ones; IMHO spadmin or setup startup performance is not critical,
 > while
 >    soffice with the usual invocations is)
 > into libOpenOffice.so and link it into soffice etc. programs
 > statically.
Uhmm, I will comment on this later (maybe tomorrow).

 > That way you kill lots of duplication (which means less relocations,
 > less
 > memory used and fewer dynamic symbols) and also prelink(8) can speed
 > it
Where can I get prelink ?



 >
 >>Another point which I am interested in is that since we are using gcc
 >>3.x for building, loading shared libraries lazy is significantly
 >>slower,
 >>one main reasons seems to be that unused symbols get relocated with
 >>gcc3
 >>which have not got relocated with gcc2. When I use nm on both
 >>libraries,
 >>the output (beside name mangling of course) looks similar. Does anyone
 >>have an idea, what this is caused by ?
 >
 >
 > The C++ ABI changed and unfortunately it is not exactly startup time
 > friendly :(.
Is there some simple explanation which can be given in a few words. I 
e.g. can imagine, that every virtual function needs to be relocated or so ?


Joerg

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: shlload.c
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20021024/ade22d17/attachment.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lst
Type: application/x-java-vm
Size: 824 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20021024/ade22d17/attachment.bin>