Re: [PATCH] Support -z initfirst for multiple shared libraries

[Yury Gribov <y dot gribov at samsung dot com>]

On 04/26/2016 11:58 PM, d wk wrote:
> Hello libc developers,
>
> In a project of mine, I needed to run some code before any constructors
> from any system libraries (such as libc or libpthread). The linker/loader
> -z initfirst feature is perfect for this, but it only supports one shared
> library. Unfortunately libpthread also uses this feature (I assume the
> feature exists because pthread needed it), so my project was incompatible
> with libpthread.
>
> So, I wrote a small patch which changes the single dl_initfirst variable
> into a linked list. This patch does not change the size of any data
> structures (it's ABI compatible), just turns dl_initfirst into a list. The
> list is not freed (the allocator wouldn't free it anyway), and insertion
> into the list is quadratic, but I expect there will never be more than
> a handful of initfirst libraries!
>
> This patch records initfirst libraries in load order, so LD_PRELOAD
> libraries will have their constructors called before libpthread. If the
> opposite behaviour is desired, the LD_PRELOAD'd library can always declare
> a dependency on libpthread. Normally LD_PRELOAD constructors are run last,
> which is very inconvenient when trying to inject new functionality, and
> I expect anyone using -z initfirst with LD_PRELOAD to really want to run
> first. The patch is written against latest glibc 2.23 (I also tested on
> glibc 2.21, and it's not quite compatible with 2.17 since the other data
> structures changed).
>
> I was not the first person to run into this problem,
> someone wanted the same thing on stack overflow two years
> ago. You can see my answer there with a complete test case.
> http://stackoverflow.com/questions/19796383/linux-ld-preload-z-initfirst/36143861
>
> Hope you will accept this patch. Comments welcome. Thanks,

Hi,

I think that many debugging/profiling tools would want this feature 
(e.g. for AddressSanitizer 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56393#c9). But here are few 
questions you may want to consider:
* is your solution compatible with DF_1_INITFIRST behavior on Solaris?
* should DF_1_INITFIRST also influence destruction order?
* what if initfirst library has some dependencies e.g. it needs malloc 
from Glibc or dlsym from libdl.so during construction (that's e.g. 
AddressSanitizer's case)? The current logic of initfirst is rather 
primitive as it does not track such dependencies at all.

-Y

> ~ dwk.
>
>
> ----[ cut here ]----
> Support -z initfirst for multiple shared libraries (run in load order).
>
> This is particularly useful when combined with LD_PRELOAD, as it is then
> possible to run constructors before any code in other libraries runs.
> ---
>   elf/dl-init.c              |  9 ++++++++-
>   elf/dl-load.c              | 19 ++++++++++++++++++-
>   elf/dl-support.c           |  4 ++--
>   sysdeps/generic/ldsodefs.h |  7 +++++--
>   4 files changed, 33 insertions(+), 6 deletions(-)
>
> diff --git a/elf/dl-init.c b/elf/dl-init.c
> index 818c3aa..da59d1f 100644
> --- a/elf/dl-init.c
> +++ b/elf/dl-init.c
> @@ -84,7 +84,14 @@ _dl_init (struct link_map *main_map, int argc, char
> **argv, char **env)
>
>     if (__glibc_unlikely (GL(dl_initfirst) != NULL))
>       {
> -      call_init (GL(dl_initfirst), argc, argv, env);
> +      struct initfirst_list *initfirst;
> +      for(initfirst = GL(dl_initfirst); initfirst; initfirst = initfirst->next)
> +        {
> +          call_init (initfirst->which, argc, argv, env);
> +        }
> +
> +      /* We do not try to free this list, as the memory will not be reclaimed
> +         by the allocator unless there were no intervening malloc()'s.  */
>         GL(dl_initfirst) = NULL;
>       }
>
> diff --git a/elf/dl-load.c b/elf/dl-load.c
> index c0d6249..1efabbf 100644
> --- a/elf/dl-load.c
> +++ b/elf/dl-load.c
> @@ -1388,7 +1388,24 @@ cannot enable executable stack as shared object
> requires");
>
>     /* Remember whether this object must be initialized first.  */
>     if (l->l_flags_1 & DF_1_INITFIRST)
> -    GL(dl_initfirst) = l;
> +    {
> +      struct initfirst_list *new_node = malloc(sizeof(*node));
> +      struct initfirst_list *it = GL(dl_initfirst);
> +      new_node->which = l;
> +      new_node->next = NULL;
> +
> +      /* We append to the end of the linked list. Whichever library was loaded
> +         first has higher initfirst priority. This means that LD_PRELOAD
> +         initfirst overrides initfirst in libraries linked normally.  */
> +      if (!it)
> +        GL(dl_initfirst) = new_node;
> +      else
> +        {
> +          while (it->next)
> +            it = it->next;
> +          it->next = new_node;
> +        }
> +    }
>
>     /* Finally the file information.  */
>     l->l_file_id = id;
> diff --git a/elf/dl-support.c b/elf/dl-support.c
> index c30194c..d8b8acc 100644
> --- a/elf/dl-support.c
> +++ b/elf/dl-support.c
> @@ -147,8 +147,8 @@ struct r_search_path_elem *_dl_all_dirs;
>   /* All directories after startup.  */
>   struct r_search_path_elem *_dl_init_all_dirs;
>
> -/* The object to be initialized first.  */
> -struct link_map *_dl_initfirst;
> +/* The list of objects to be initialized first.  */
> +struct initfirst_list *_dl_initfirst;
>
>   /* Descriptor to write debug messages to.  */
>   int _dl_debug_fd = STDERR_FILENO;
> diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
> index ddec0be..198c089 100644
> --- a/sysdeps/generic/ldsodefs.h
> +++ b/sysdeps/generic/ldsodefs.h
> @@ -326,8 +326,11 @@ struct rtld_global
>     /* Incremented whenever something may have been added to dl_loaded.  */
>     EXTERN unsigned long long _dl_load_adds;
>
> -  /* The object to be initialized first.  */
> -  EXTERN struct link_map *_dl_initfirst;
> +  /* The list of objects to be initialized first.  */
> +  EXTERN struct initfirst_list {
> +    struct link_map *which;
> +    struct initfirst_list *next;
> +  } *_dl_initfirst;
>
>   #if HP_SMALL_TIMING_AVAIL
>     /* Start time on CPU clock.  */