This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: atexit, __cxa_atexit and __cxa_finalize interaction



On 11/06/2019 16:45, Carlos O'Donell wrote:
> On 6/11/19 3:06 PM, Adhemerval Zanella wrote:
>> Recently on libc-help [1], a user reported it observed some issues regarding
>> atexit order calling in some scenarios (callbacks were not issued in the 
>> expected order for some scenarios). After some time, he could provide an 
>> actual testcase that shows the issue he was referring [2].
> 
> This issue has been around since the dawn of time.
> 
> https://freebsd-arch.freebsd.narkive.com/zzyHX3Bw/dlclose-vs-atexit
> 
> The options as outlined in the old FreeBSD discussion are informative.
> 
> (1) Run atexit() at exit() after dlclose() and crash.
> (2) Run atexit() at dlclose() and try not to crash (impossible for the cyclic
>     case where two libraries call eachothers atexit functions i.e. no total
>     order possible, and we could detect this).
> (3) Have atexit() increment the library ref count, and decrement it after
>     it runs during exit(), making the library unloadable (what SunOS does?)

We can also do not run atexit on dlclose and also remove DSO registered ones
handlers from internal list.  It makes more sense to me that once you unload
a library, you potentially remove all associated runtime mechanism associated
with it.

> 
> My opinion is this:
> 
> (a) glibc promises to try and unload your DSO on dlclose().
> 
> (b) glibc has always called atexit() on dlclose(), because the intent of
>     atexit() for a DSO was to handle the case where it's going away and
>     you want to cleanup state. It minimizes the memory needed to store the
>     list of these calls, and fails early, both are good.

I think we might have better options here:

  - gcc already provides destructors, which will be called by __cxa_finalize.

  - We can also provide a gnu extension (such as we did for on_exit) which 
    essentially register an handler as __cxa_atexit.

> 
> (c) LSB is a trailing standard that needs to document what GNU/Linux
>     defines as the standard.
> 
> I don't think there is anything we need to do to document this better in
> our manual.
> 
> The linux man pages already say:
> 
>   Linux notes
>        Since  glibc 2.2.3, atexit() (and on_exit(3)) can be used within a shared library to establish functions that
>        are called when the shared library is unloaded.
> 
> This is a perfectly acceptable use case for dlclose () IMO.
> 
>> After some debugging, it turned out to be how atexit, __cxa_atexit and
>> __cxa_finalize are defined [3]. In a short:
>>
>>   - __cxa_atexit register a function to be called by exit or when a shared 
>>     library is unloaded. 
>>
>>   - __cxa_finalize is called on library unload (either when a program is exiting
>>     or by a dlopen)
>>
>>   - atexit is implemented by calling __cxa_atexit through a static object
>>
>> It then makes atexit acts exactly as __cxa_atexit, which makes unload shared 
>> libraries to call atexit handlers are well to __cxa_atexit handlers.
>>
>> As on my later findings [4], it seems to come from LSB definition (although
>> I am not sure which one comes first, the standard or the implementation). 
>>
>> In any case, I do think that making atexit works as __cxa_atexit is problematic
>> for glibc, since it does try to actually unload the shared libraries on dlclose.
>>
>> POSIX [5] standard does not really specify the dlclose interaction with atexit 
>> handlers, so I think we might assume it is an implementation detail. And by 
>> making atexit handlers being called on __cxa_finalize it makes then run in 
>> non-expected order when they are registered in shared libraries constructors.
>>
>> So the question is anything use preventing to change __cxa_finalize to no handle 
>> atexit handlers, only __cxa_atexit, and quick_exit ones)?
>>
>> If we could, my idea would be to:
>>
>>   1. Either add an atexit symbol or an __atexit (since currently atexit
>>      is build statically) which add a ef_at exit_function entry on __exit_funcs.
>>      Old binaries would still call __cxa_atexit, so we do not actually
>>      need to add a compat symbol.
>>
>>   2. Make __cxa_finalize set ef_at handlers to ef_free to avoid them being
>>      called later (similar to quick_exit).
> 
> What is the use case for making the chnage?

To align better for what is expected from both atexit (run handler at *exit*) 
and C definition of handlers ordering, which as you noted caused not only 
confusion but I think it also might be interpreted as standard violation.  

I think this extra functionality that make atexit and __cxa_atexit semantically 
similar is what adds complexity here. 

The __cxa_atexit does make sense to run on library unloading, to allow C++ 
code to call destructors for library objects. But I can't really see real
gains of extending it to atexit as well (now that we have shared library
destructors).

> 
> New developers do stumble over this issue, but it's a complex issue and we
> could do better to document it, like including Ben's patch on dynamic loading:
> 
> https://www.sourceware.org/ml/libc-alpha/2017-10/msg00582.html
> 
> So we have good docs to read.
> 
>> The only issue I can think of is c++ compilers that might rely on atexit
>> to works as __cxa_atexit, however __cxa_atexit is provided by glibc
>> since forever so I don't think it should be a problem.
>>
>> [1] https://sourceware.org/ml/libc-help/2019-05/msg00021.html
>> [2] https://github.com/mulle-nat/ld-so-breakage
>> [3] https://sourceware.org/ml/libc-help/2019-06/msg00008.html
>> [4] https://sourceware.org/ml/libc-help/2019-06/msg00009.html
>> [5] https://pubs.opengroup.org/onlinepubs/9699919799/functions/atexit.html
>>
> 
> 


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]