[RFC] Prevent tailcall optimizations of libdl functions

[Florian Weimer]

On 01/25/2017 09:12 AM, Yuri Gribov wrote:
> Hi all,
>
> This is a ping on https://sourceware.org/bugzilla/show_bug.cgi?id=21050
>
> Some libdl functions rely on return address to figure out the calling
> DSO and then use this information in computation (e.g. output of dlsym
> depends on which library called it).
>
> As reported in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66826 this
> may break under tailcall optimization i.e. in cases like
>
>   return dlsym(...);
>
> Rich Felker suggested that adding new GCC attribute is a possible
> solution.

Several functions have essentially the same issue, so this seems a 
reasonable approach.

> Another option would be to hack around dlsym to prevent
> tailcall from happening e.g.
>   #define dlsym(h, name) { volatile void *sym = dlsym(h, name); return
> (void *)sym; }

I'm not convinced that this prevents a tail call in all cases.  I'd also 
caution against adding yet another mechanism to preserve the caller context.

If we cannot get the GCC attribute to work, we should move dlsym into 
libc_nonshared.a and use __dso_handle, like we do for other functions 
which require the caller context.  This is not the direction I want us 
to move in; I prefer __builtin_return_address because it is easier to 
maintain, but that has the problem with tail calls.

Thanks,
Florian