Re: Why does elf/dl-load.c have local_strdup?

[Roland McGrath <roland at hack dot frob dot com>]

> I'm fixing this partly because I've just found that ld.so.cache
> management has no ref counting on the mmap, and recursive dlopen leads to
> segfaults if you have a malloc hook that calls dlopen. The deepest
> dlopen, as it completes, unmaps the cache, and the next deepest dlopen
> segfaults trying to reference data in the cache it expects to be valid.

Understood.

> I figure that malloc hooks should be allowed to call dlopen, and we should
> either ref-count ld.so.cache mmap, or better yet, just change the contract
> of _dl_load_cache_lookup to return a copy of the caches string.

That sounds harmless on its face, but it is clearly the edge of a slippery
slope.  What freedom of action should malloc hooks really expect?  Library
code calls malloc from all sorts of deep internal places.  It's not
reasonable to reorganize all our code paths so that they can be reentered
safely from malloc.  You've found one case and proposed one workaround, for
a single specific direct path of reentry.  There are lots of other paths,
and a huge combinatoric explosion of indirect paths of reentry.  Should a
malloc hook be able to call printf?  What about dlopen->malloc->printf?
What about printf->malloc->dlopen?  

Why should we support dlopen->malloc->dlopen any more than we support
dlopen->signal handler->dlopen?

> In 99% of the caches the hot path is that the cache has the file, and
> you're going to strdup anyway, so why not just do it while you know the
> cache to be valid and avoid this entire issue? Well, one issue is that any
> strdup in _dl_load_cache_lookup will trigger the problem (call malloc,
> which calls a malloc hook, which calls dlopen, which later unmaps the
> cache you're trying to copy). So you'd need an intermediate alloca,
> followed by a strdup.

What you mean is an intermediate strdupa to copy the value before doing a
strdup.  When talking about cost, it's important to be precise.  It's the
CPU cost of alloca (constant) plus the cost of strlen+memcpy (roughly
linear in the length of the file name), plus the other kinds of cost of
deepening the stack by a variable amount.

> Is `alloca + strdup` worse or better than `ref-counted ld.so.cache mapping?`

The somewhat open-ended deepening of the stack concerns me more than the
CPU cost of strdupa.  (This is all in preparation for open, mmap, reloc
processing with COW faults, etc., so the actual CPU cost of strdupa will
pale in comparison.)

But the slippery slope concerns me most of all.