Re: [RFC] stdlib: Make atexit to not act as __cxa_atexit

[Adhemerval Zanella <adhemerval dot zanella at linaro dot org>]

On 08/07/2019 08:16, Florian Weimer wrote:
> * Adhemerval Zanella:
> 
>> For _dl_fini, we already have the link_map for object that calls 
>> __cxa_finalize. What we need is to filter out the exit handlers registered
>> by the object itself, so its calls only the functions registered by the 
>> shared library referenced by the link_map.
>>
>> Not sure how easily we can accomplish it on exit handlers registration 
>> functions (the __dso_handler trick is to make this easier).
> 
> We can find the object that contains the address of __dso_handle, either
> at registration time (perhaps better, to keep dlclose cost lower), or
> during _dl_fini.

We currently have the following:

  (libdl) dlclose:
          \_ (ld.so) _dl_close_worker
		     - run dt_fini_array
                       \_ (libfoo.so) __do_global_dtors_aux
                                      \_ (libc.so) __cxa_finalize ((libfoo.so) __dso_handle)

What you are suggesting, if I understood correctly is:

  (libdl) dlclose:
          \_ (ld.so) _dl_close_worker 
                     \_ (libc.so) __cxa_finalize ((libfoo.so) __dso_handle)

So what we need to get on _dl_close_worked is the (libfoo.so) __dso_handle
value. I do think it is feasible, we can  add a new field on the link_map to 
hold its value or find it on _dl_close_worker. We will need to handle some 
caveats, as to add symbol resolution for local symbols and handle libraries 
that does not have __do_global_dtors_aux (and thus no __dso_handle).

However I do think we can fix it by tune how to organize the atexit internal
lists.

> 
> 
>> The idea I am exploring is to change on how new handlers are included in
>> the exit handler lists, so on a default execution (without handler adding
>> new entries) atexit/on_exit are executed before __cxa_atexit.
> 
> Sorry, what do you mean?  That *handlers* registered using
> atexit/on_exit are executed before those registered via __cxa_atexit?

My idea is to organize the internal list to keep the atexit/on_exit handler
before __cxa_atexit, so on both __cxa_finalize and exit/quick_exit they will
be executed first.

> 
>> The change is to make exit_function_list have an associated type which
>> hold an only a specific set of handlers.  Currently we required just
>> two types, one for ef_at/ef_on handlers and one for ef_cxa. New inclusion 
>> are added in a ordered manner, where ef_at/ef_on are added on 
>> exit_function_list always before ef_cxa.  
>>
>> Although on __cxa_finalize we will also need to explicit handle ef_at as
>> well (for dlclose), the idea is also to mimic the ef_cxa behaviour and set
>> the ef_at/ef_on free after its execution to avoid multiple executions 
>> (as a side note I think by not setting ef_on currently we might run on_exit 
>> handlers twice in some scenarios).
> 
> For dlclose, we also need to perform real deallocation at a certain
> point, to avoid memory leaks.
> 
> Have you considered a design which puts the same handler entry on two
> different lists?

Yes, the problem is we need to handle multiple constraints:

  - Any function that register a new exit handle may be called from another
    one. For instance atexit my call atexit or __cxa_atexit may call atexit
    (or any combination).

  - The internal lists follow the GNU principle of adding no limit, so we
    support the minimal POSIX states using a static allocated initial list
    (so atexit can't fail due malloc failure), and we add a linked list
    to make it grow as requested.

  - We need to handle BZ#14333, so we need to stop new registered function
    once we are over iterating over the list. I don't think it would be
    possible with two lists, specially for the case where one callback
    can modify the another list.