[RFC] Prevent tailcall optimizations of libdl functions

[Florian Weimer]

On 01/26/2017 04:04 AM, Carlos O'Donell wrote:
> On 01/25/2017 06:28 AM, Florian Weimer wrote:
>> I'd also caution against adding yet another mechanism to preserve the
>> caller context.
>
> Would you mind expanding a bit on your caution here?

I'm concerned about adding more and more magic, especially when the 
magic ends up causing code generation changes and is not restricted to 
shared objects.

For example, what happens with JIT compilation, where the caller address 
is not within code which was loaded by the dynamic linker?

Maybe we should just change the definition of RTLD_DEFAULT and 
RTLD_NEXT, to something like this:

#define RTLD_DEFAULT ((void *) ((unsigned long) &__dso_handle | 1)))
#define RTLD_NEXT ((void *) ((unsigned long) &__dso_handle | 3)))

Even more explicit would be if we made __dso_handle a real void * 
variable pointing to the relevant link map, not just something that has 
a unique address, so that we'd save the reverse lookup.  We could even 
have real handles representing the contexts, but that might be overkill.

Looking at the implementation, we do not need the caller information if 
the specified handle is a real handle and neither RTLD_DEFAULT nor 
RTLD_NEXT, so changing the definition of those macros accurately 
reflects what is actually going on in the implementation.

Thanks,
Florian