Getting offset of inital-exec TLS variables on GNU/Linux
Carlos O'Donell
carlos@redhat.com
Fri May 24 20:12:00 GMT 2019
On 5/24/19 11:23 AM, Florian Weimer wrote:
> * Simon Marchi:
>
>> On 2019-05-17 10:34 a.m., Florian Weimer wrote:
>>> * Simon Marchi:
>>>
>>>> On 2019-05-17 6:21 a.m., Florian Weimer wrote:
>>>>> Is it possible to obtain the offset of initial-exec TLS variable on
>>>>> GNU/Linux?
>>>>>
>>>>> It doesn't seem so because GDB executes the DWARF to access the TLS
>>>>> variable, so the offset is an implementation detail. Although it is
>>>>> often visible at the ELF layer.
>>>>>
>>>>> Thanks,
>>>>> Florian
>>>>
>>>> Can you clarify a little bit?
>>>>
>>>> You are looking for the offset the variable from which point of reference:
>>>>
>>>> - The start of the TLS area of this module?
>>>> - The start of the TLS area of the current thread?
>>>
>>> The offset from something related to the thread pointer to the variable,
>>> for cases where this is constant (specifically, initial-exec TLS
>>> variables).
>>>
>>>> Also, are you looking for something you can find statically, with just the
>>>> executable, or are you working in the context of the live process?
>>>
>>> I have a live process, and it would be best if the information matched
>>> that process (even if it uses different libraries than those currently
>>> installed in the file system).
>>
>> Hi Florian,
>>
>> I am still a bit unsure of what you are looking for concretely. Are
>> you looking for a GDB command to print this offset? Do you need to
>> compute the offset in an external tool? what information are you
>> starting with? I think that a bit more context would help us help you.
>
> I had hoped to get this offset so that I can access the TLS variable
> without loading libthread_db.
>
> I'd be fine with getting the module TLS offset from internal data
> structures.
>
>> GDB uses libthread_db to get the location of TLS variables, which
>> leaves all implementation details about this to glibc. And since you
>> are a glibc maintainer, I am not too sure what I can teach you about
>> this, as you probably know more about this than I do.
>
> If I have libpthread_db loaded and can somehow get the thread pointer
> (e.g., $fs_base on x86-64), then I can get the constant I want just by
> computing the different between the two, like this:
>
> (gdb) print (void *)&tcache - (void *)$fs_base
> $9 = -80
>
> Going back a bit, I'm not sure what the API contract is for
> DW_OP_GNU_push_tls_address. It's not really clear to me if under the
> ELF TLS ABI, there is an expectation that the dynamic linker always
> allocates the TLS space for a DSO as a single block. I've perused the
> two documents for the GNU ELF TLS ABI, and this is never spelled out
> explicitly.
You are correct, it is not spelled out that the space for the loaded
module needs to be in a single contiguous region of memory, but it is
implied by the design of PT_TLS.
In practice we have no gaps because the initialization image for the block,
particularly for thread-local initialized global data, is copied as one
continuous block. If we wanted to support gaps we would need a more complex
definition than just the PT_TLS marker we use to identify that region (and
which sections go into that region during static link).
Does this answer your question about why it needs to be a single
continuous block?
Which two documents did you review?
> Outside debugging information, a TLS relocation for non-initial-exec TLS
> always consists of a pair of a module ID and an offset. Therefore, it
> should be possible to lazily allocate individual TLS variables within a
> DSO, by assigning them separate module IDs.
You should be able to do that, but for each variable you must have access
to the size via symtab st_size to be able to copy the potentially initialized
value from PT_TLS, otherwise the block must be the full size and you must
initialize it as if it were all the data for the DSO.
> But what seems to happen in practice is that there is just one TLS block
> per DSO, which is allocated at once once the first TLS variable is
> accessed. Furthermore, the entire TLS block is allocated non-lazily if
> there is a single initial-exec TLS variable in a DSO.
Just to be clear there is one TLS block per DSO per thread, which is allocated
once.
Yes, the entire TLS block is allocated non-lazily, and it's initializing image
is copied from PT_TLS for that module.
Once you touch any TLS variable for the DSO the whole DSO is initialized for
use.
Yes, any IE TLS vars force the allocation of the whole block because there could
be IE TLS var uses immediately after startup relocation processing of the TPOFF
GOT relocations.
> Based on that, I
> conclude that the module IDs are used only to share TLS variables for
> the same symbol across multiple modules.
I do not think this conclusion is accurate.
What you are observing is a consequence of the fact that the static linker,
during construction of the binary, optmizes all DSOs seen at static link time
and places them into the initial TLS block. And as many of the references are
optimized into TLS IE, which have no module ID at all (they don't need it).
The module ID is intended to reference the module or DSO, to allow the memory
for that module be loaded lazily.
> Due to this restriction, the
> module ID for a TLS variable can be inferred from the object that
> contains the DW_OP_GNU_push_tls_address opcode (and hopefully locating
> the object based on symbol name in the debugger matches the way dynamic
> linker search symbols).
I don't understand what you mean by "restriction" here?
The module ID for a TLS variable is assigned at runtime, and any inspection
process with a live process could find the module ID for the module by
looking for DTPMOD relocations, and reading their values out of the GOT
for the DSO.
In practice it's one mod id per DSO (but as you note above it need not be,
but is restricted by PT_TLS design). So yes, if you see a DSO and it has
a DW_OP_GNU_push_tls_address, you can determine it's module ID by inspecting
the DSOs GOT given dynamic relocation information, and once you have the
module ID you can call __tls_get_addr with the symbol offset to get the final
address (or find the dtv and traverse it for the thread).
I don't actually understand how DW_OP_GNU_push_tls_address works, but the
comments for it seem to indicate it's a hack that glibc is supposed to fix,
but I've never been asked about it :-)
It *looks* like DW_OP_GNU_push_tls_address is just the offset from the start
of the block, but that's still not enough to compute the final address of
the variable in memory.
> I wonder if all this implicitly but firmly encodes a correspondence
> between the argument to DW_OP_GNU_push_tls_address and the offset of a
> TLS variable, within the TLS block for a DSOâif the DSO has the
> DF_STATIC_TLS flag set.
Yes, I think this is true for DF_STATIC_TLS, because all such DSOs share
the same initial TLS block in the exectuable, so they are all offsets
from the same block.
I do see DWARF5 has DW_OP_form_tls_address, which is effectively the way
to translate a runtime specific argument + a TLS variable offset into a
final address.
Is DW_OP_form_tls_address the DWARF5 generic equivalent of
DW_OP_GNU_push_tls_address?
> Asumming this is indeed true, we could add the
> TLS offset of a DSO to the public part of the link map in the glibc
> dynamic loader to help debuggers.
I wouldn't call it the TLS offset of a DSO, instead I would call it
what it is "static TLS offset", and if DF_STATIC_TLS is set for that
map, then you know you can find all the TLS variables without libthread_db.
> For TLS variables defined in
> DF_STATIC_TLS DSOs, it should then be possible to access the TLS
> variable without the help of libthread_db, assuming that we teach GDB
> how to compute the TLS variable address from the thread pointer, DSO TLS
> offset, and variable offset (something binutils seems to know for
> several targets already, to implement relaxations).
Right, you would need:
TP + DSO offset + VAR offset = final symbol address.
The VAR offset is actually the DW_OP_const<n><x> operations prior to the
DW_OP_form_tls_address call.
TP + DSO offset is 'td_thr_tlsbase' from nptl_db, so you would be exposing
only the DSO offset value in the link map *only* for the DF_STATIC_TLS case,
because it's a fixed value that won't change.
Note that 'td_thr_tls_get_addr' from nptl_db does the above calculation.
We'd be teaching gdb how to manually run 'td_thr_tlsbase' for the limited
case of DF_STATIC_TLS. It would solve some of the problems we have, and
remove any hacks for errno, but would not fully solve dynamic TLS variable
access in a single threaded program (though it's a big step forward).
> Would this be a reasonable thing to do?
>
> (Cc:ing Carlos, who probably knows what is really going on here.)
I think your conclusion is sound, even if not all the intermediate steps
made sense to me.
--
Cheers,
Carlos.
More information about the Gdb
mailing list