Problem with atexit and _dl_fini

[Nat!]

Another datapoint to support my claim that _dl-fini breaks atexit. This 
time its very easy to reproduce ;)

Here 's the README.md from the Github Repo 
https://github.com/mulle-nat/atexit-breakage-linux


```

# Shows another breakage involving `atexit` on linux

Here the `atexit` callback is invoked mistakenly multiple times.


## Build

Build with [mulle-make](//github.com/mulle-sde/mulle-make) or alternatively :

```
(
    mkdir build &&
    cd build &&
    cmake .. &&
    make
)
```

## Run

Use `ldd` to trigger the misbehaviour:

```
LD_PRELOAD="${PWD}/build/libld-preload.so" ldd ./build/main
```

## Output

```
load
unload
unload
unload
    linux-vdso.so.1 (0x00007ffd2b2bd000)
    /home/src/srcO/mulle-core/mulle-testallocator/research/ld-preload/build/libld-preload.so (0x00007f83853c1000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f838518c000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f83853cd000)
unload
unload
```


Ciao
    Nat!


On 19.05.19 21:37, Nat! wrote:
>
> On 19.05.19 18:23, Florian Weimer wrote:
>> * Nat!:
>>
>>> So my problem is, that I observe that my atexit calls are not executed
>>> in the correct order.
>>>
>>> i.e. atexit( a); atexit( b);Â  should result in b(), a() being called in
>>> that order. To quote the man page,
>>>
>>> "the registered functions are invoked in reverse order".
>>>
>>>
>>> When I register with `atexit` I can see my functions being added
>>> properly within `__internal_atexit` in the
>>>
>>> correct order. Finally after my functions, the elf-loader ? also adds
>>> itself there. So it is being called first by
>>>
>>> `__run_exit_handlers`.
>>>
>>>
>>> Then comes the part where it goes wrong. I registered my two function
>>> with `__internal_atexit`, but for some reason
>>>
>>> `_dl_fini` is calling `__cxa_finalize` and that is calling the wrong
>>> function first.
>> When atexit is called from a DSO, glibc calls the registered function
>> before the DSO is unloaded.  This choice was made because after
>> unloading, the function pointer becomes invalid.
>>
>> I haven't checked, but I suspect atexit still works this way even if
>> it doesn't have to (because the DSO is never unloaded).
>>
>
> I understand, but the behavior is wrong :) The C standard (or the C++ 
> standard for this matter) 
> http://www.cplusplus.com/reference/cstdlib/atexit/ states that
>
>
> ```
>
> If more than one atexit function has been specified by different calls 
> to this function, they are all executed in reverse order as a stack 
> (i.e. the last function specified is the first to be executed at exit).
>
> ```
>
> I think its been shown that glibc can violate this C standard, so for 
> me the argument would be over here already. That one should unwind in 
> the reverse order is, I assume, not a interesting discussion topic. 
> Currently atexit as a reliable mechanism is broken.
>
>
> But I also don't think the way this is currently handled in glibc, 
> can't be of much use to anyone.
>
> Case 1: a regular exe linked with shared libraries, nothing gets 
> unloaded at exit, so what's the point ?
>
> Case 2: someone manually unloads a shared library, that contains 
> atexit code. The bug is either using `atexit` for a shared library 
> that gets unloaded, or unloading a shared library that contains atexit 
> code. But it's not really glibcs business IMO.
>
> Case 3: some automatism unloads shared libraries. Then the automatism 
> should check if atexit code is affected and not unload, because the 
> shared library is still clearly needed. It's a bug in the automatism.
>
> If one was hellbent on trying to support atexit for unloading shared 
> libraries, an atexit contained in a shared library should up the 
> reference count of the shared library during the atexit call and 
> decrement after the callback has executed.
>
> Ciao
>
> Â Â  Nat!
>
>