Re: Speeding up the dynamic linker with 100s of DSOs?

[michael meeks <michael dot meeks at novell dot com>]

Hi Andrew,

	So - we have the same problem in OO.o - but we have 'only' 150DSOs
loaded instead of 300 almost all dlopened.

	I have several patches addressing this posted to the binutils list -
particularly implementing -Bdirect linking (which stores the
compile-link-time information as to which library a symbol is defined
in) so that we can look there 1st at run-time. That of course, removes
75% or so of the O(num-libraries) lookups from the link[1].

	Of course - a facile implementation of -Bdirect would have serious
problems with C++ - because of the vague linkage / ODR issues - so, the
patch detects which symbols are vague during library compile/link, then
propagates that information to other libraries, so life is good.

	In addition; by sorting relocations & dynsym & dynstr entries one can
get another ~25% win, and by storing & comparing pre-computed hash
values one can get a ~50% win - so, all in all if we could get this
stuff into general glibcs[2] ;-), performance of large, modularized C++
apps would get a lot better. [ there are 2 final, more maginal tweaks
queued pending this getting rationalized too :-].

	Check:

-Bdirect:	http://sourceware.org/ml/binutils/2005-11/msg00380.html
sorting:	http://sourceware.org/ml/binutils/2006-01/msg00024.html
.hashvals	http://sourceware.org/ml/binutils/2006-01/msg00171.html

	The latest patches .suse.ized are at:

	http://go-oo.org/ooo-build/patches/test/*suse*.diff

	As one blogs about the work, I've had the Troll-tech guys & a random
ISV play with various features - suffering the same problems. I'd love
to know what your experience of testing these is - that is if you are ok
with re-building your glibc :-)

	HTH,

		Michael.

[1] - the other 25% of relocations in my case being vague type info.
[2] - SL 10.1 should ship with some of this, though it's not there yet.
-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot