[RFC] Prevent tailcall optimizations of libdl functions

[Florian Weimer]

On 01/25/2017 04:50 PM, Jakub Jelinek wrote:
> On Wed, Jan 25, 2017 at 04:43:32PM +0100, Florian Weimer wrote:
>> On 01/25/2017 04:41 PM, Szabolcs Nagy wrote:
>>> On 25/01/17 15:40, Florian Weimer wrote:
>>>> On 01/25/2017 04:38 PM, Szabolcs Nagy wrote:
>>>>> On 25/01/17 15:32, Yuri Gribov wrote:
>>>>>> FWIW it sounds like GCC attribute would be the most natural solution
>>>>>> (and probably also useful in other contexts). I'll try to cook a patch
>>>>>> for GCC if there are no objections.
>>>>>
>>>>> note that even if dlsym is marked notailcall, with
>>>>>
>>>>> p = dlsym;
>>>>> ...
>>>>> return p();
>>>>>
>>>>> if the type of p does not carry the attributes of
>>>>> dlsym then this can be a tailcall.
>>>>
>>>> Yes, but dlsym would still see the address of the function containing the dlsym call, which is what we need to
>>>> determine the relevant object.
>>>
>>> how?
>>
>> At worst, the return address points to the code which jumps to the address
>> p.  This code is still in the object which calls dlsym.
>
> That can be a tail call and you get exactly the same problem as with direct
> calls to dlsym.

Sorry, I don't see how.  On x86_64, we might end up with this 
instruction sequence:

   call dlsym
   jmp %rax

But this means that the call to dlsym is not a tail call, and 
__builtin_return_address in dlsym will return the address of the jmp 
instruction.

A tail call has to be in a tail position.  If there is another function 
call after it, it is no longer in a tail position.

What am I missing?

Thanks,
Florian