Bug 23293 - aarch64: getauxval is broken when run as ld.so ./exe and ld.so adjusts argv on the stack
Summary: aarch64: getauxval is broken when run as ld.so ./exe and ld.so adjusts argv o...
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: dynamic-link (show other bugs)
Version: 2.27
: P2 normal
Target Milestone: 2.36
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-06-15 12:51 UTC by Szabolcs Nagy
Modified: 2022-06-10 00:07 UTC (History)
5 users (show)

See Also:
Host:
Target: aarch64-*-*, arm-*-*, nios2, ia64, alpha, s390-32, sparc
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Szabolcs Nagy 2018-06-15 12:51:17 UTC
on several targets the rtld entry point (_dl_start_user) adjusts
argv on the stack to have it right for the executed application
and to avoid a misaligned sp in the application elf entry point.

getauxval seems to use a global GLRO(dl_auxv) value that is
computed before the adjustments.

affects at least aarch64 and arm.

$ cat a.c
#include <stdio.h>
#include <sys/auxv.h>

int main()
{
	for (int i=0; i < 40; i++)
		printf("%2d 0x%lx\n", i, getauxval(i));
}
$ ./a.out 
 0 0x0
 1 0x0
 2 0x0
 3 0xaaaad59e5040
 4 0x38
 5 0x8
 6 0x1000
 7 0xffff9c5fd000
 8 0x0
 9 0xaaaad59e5670
10 0x0
11 0x3e8
12 0x3e8
13 0x3e8
14 0x3e8
15 0xffffce3884b8
16 0x807
17 0x64
18 0x0
19 0x0
20 0x0
21 0x0
22 0x0
23 0x0
24 0x0
25 0xffffce3884a8
26 0x0
27 0x0
28 0x0
29 0x0
30 0x0
31 0xffffce388ff0
32 0x0
33 0xffff9c628000
34 0x0
35 0x0
36 0x0
37 0x0
38 0x0
39 0x0
$ /lib/ld-linux-aarch64.so.1 ./a.out 
 0 0x0
 1 0x0
 2 0x0
 3 0x0
 4 0x0
 5 0x0
 6 0x0
 7 0x0
 8 0x7
 9 0x0
10 0x0
11 0x0
12 0x0
13 0x0
14 0x0
15 0x0
16 0x807
17 0x0
18 0x0
19 0x0
20 0x0
21 0x0
22 0x0
23 0x0
24 0x0
25 0x0
26 0x0
27 0x0
28 0x0
29 0x0
30 0x0
31 0x0
32 0x0
33 0x0
34 0x0
35 0x0
36 0x0
37 0x0
38 0x0
39 0x0
Comment 1 Szabolcs Nagy 2018-06-15 13:56:35 UTC
i think it affects all targets with DL_ARGV_NOT_RELRO == 1
Comment 2 Michael Hudson-Doyle 2020-06-24 21:16:40 UTC
I've just run into this because it causes a glibc test failure when libnss-systemd is enabled (https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1869364).

Would a fix be for _dl_start_user to update _dl_auxv in the same way it updates _dl_argv? Or is that just naive (start up code confuses me!)
Comment 3 Florian Weimer 2020-06-25 08:52:32 UTC
(In reply to Michael Hudson-Doyle from comment #2)
> I've just run into this because it causes a glibc test failure when
> libnss-systemd is enabled
> (https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1869364).
> 
> Would a fix be for _dl_start_user to update _dl_auxv in the same way it
> updates _dl_argv? Or is that just naive (start up code confuses me!)

I think this should work. There does not seems to be anything that stores the address if auxv entries, only values read from the vector.
Comment 4 Szabolcs Nagy 2020-06-25 15:34:42 UTC
(In reply to Florian Weimer from comment #3)
> (In reply to Michael Hudson-Doyle from comment #2)
> > I've just run into this because it causes a glibc test failure when
> > libnss-systemd is enabled
> > (https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1869364).
> > 
> > Would a fix be for _dl_start_user to update _dl_auxv in the same way it
> > updates _dl_argv? Or is that just naive (start up code confuses me!)
> 
> I think this should work. There does not seems to be anything that stores
> the address if auxv entries, only values read from the vector.

it may work, but i think _dl_start_user stack handling should
not be updated, it should be removed: the logic is backwards:

most targets have fragile asm for the rtld start code to shuffle
around entries on the stack, but such shuffling can be done in c
in the generic code (e.g. in dl_main).

this would also fix the (imo security) bug that the protection
of dl data is not consistent across targets (DL_ARGV_NOT_RELRO).

and all this mess is for saving a few cycles on targets where
you don't need to do the shuffling. i think the generic code
should work for all targets and those who wish to optimize
add some hacks (e.g. ifdef out the shuffling), not the other
way around.

i didnt get the chance to clean this up yet, so it's not fixed.
Comment 5 Florian Weimer 2020-06-25 15:39:46 UTC
(In reply to Szabolcs Nagy from comment #4)
> (In reply to Florian Weimer from comment #3)
> > (In reply to Michael Hudson-Doyle from comment #2)
> > > I've just run into this because it causes a glibc test failure when
> > > libnss-systemd is enabled
> > > (https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1869364).
> > > 
> > > Would a fix be for _dl_start_user to update _dl_auxv in the same way it
> > > updates _dl_argv? Or is that just naive (start up code confuses me!)
> > 
> > I think this should work. There does not seems to be anything that stores
> > the address if auxv entries, only values read from the vector.
> 
> it may work, but i think _dl_start_user stack handling should
> not be updated, it should be removed: the logic is backwards:
> 
> most targets have fragile asm for the rtld start code to shuffle
> around entries on the stack, but such shuffling can be done in c
> in the generic code (e.g. in dl_main).
> 
> this would also fix the (imo security) bug that the protection
> of dl data is not consistent across targets (DL_ARGV_NOT_RELRO).
> 
> and all this mess is for saving a few cycles on targets where
> you don't need to do the shuffling. i think the generic code
> should work for all targets and those who wish to optimize
> add some hacks (e.g. ifdef out the shuffling), not the other
> way around.
> 
> i didnt get the chance to clean this up yet, so it's not fixed.

That sounds of course very reasonable. As long as very manipulate the arguments as arrays, the C code should be very portable. An assembler stub will still be needed, but it will be much smaller.

I also think we should move the assembler code out of dl-machine.h while we are at it. We can start out with an empty dl-start.S file, and gradually move ports to follow the powerpc32 example.
Comment 6 Florian Weimer 2022-05-03 08:33:43 UTC
I believe this is causing dlfcn/tst-dlinfo-phdr to fail in some cases.
Comment 7 Sourceware Commits 2022-05-17 09:19:43 UTC
The master branch has been updated by Szabolcs Nagy <nsz@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=ad43cac44a6860eaefcadadfb2acb349921e96bf

commit ad43cac44a6860eaefcadadfb2acb349921e96bf
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Fri Jun 15 16:14:58 2018 +0100

    rtld: Use generic argv adjustment in ld.so [BZ #23293]
    
    When an executable is invoked as
    
      ./ld.so [ld.so-args] ./exe [exe-args]
    
    then the argv is adujusted in ld.so before calling the entry point of
    the executable so ld.so args are not visible to it.  On most targets
    this requires moving argv, env and auxv on the stack to ensure correct
    stack alignment at the entry point.  This had several issues:
    
    - The code for this adjustment on the stack is written in asm as part
      of the target specific ld.so _start code which is hard to maintain.
    
    - The adjustment is done after _dl_start returns, where it's too late
      to update GLRO(dl_auxv), as it is already readonly, so it points to
      memory that was clobbered by the adjustment. This is bug 23293.
    
    - _environ is also wrong in ld.so after the adjustment, but it is
      likely not used after _dl_start returns so this is not user visible.
    
    - _dl_argv was updated, but for this it was moved out of relro, which
      changes security properties across targets unnecessarily.
    
    This patch introduces a generic _dl_start_args_adjust function that
    handles the argument adjustments after ld.so processed its own args
    and before relro protection is applied.
    
    The same algorithm is used on all targets, _dl_skip_args is now 0, so
    existing target specific adjustment code is no longer used.  The bug
    affects aarch64, alpha, arc, arm, csky, ia64, nios2, s390-32 and sparc,
    other targets don't need the change in principle, only for consistency.
    
    The GNU Hurd start code relied on _dl_skip_args after dl_main returned,
    now it checks directly if args were adjusted and fixes the Hurd startup
    data accordingly.
    
    Follow up patches can remove _dl_skip_args and DL_ARGV_NOT_RELRO.
    
    Tested on aarch64-linux-gnu and cross tested on i686-gnu.
    
    Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
Comment 8 Sourceware Commits 2022-05-17 09:19:53 UTC
The master branch has been updated by Szabolcs Nagy <nsz@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9faf5262c77487c96da8a3e961b88c0b1879e186

commit 9faf5262c77487c96da8a3e961b88c0b1879e186
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Tue May 3 13:18:04 2022 +0100

    linux: Add a getauxval test [BZ #23293]
    
    This is for bug 23293 and it relies on the glibc test system running
    tests via explicit ld.so invokation by default.
    
    Reviewed-by: Florian Weimer <fweimer@redhat.com>
    Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Comment 9 Szabolcs Nagy 2022-05-17 12:28:45 UTC
fixed for 2.36.
Comment 10 Sourceware Commits 2022-05-19 09:52:03 UTC
The release/2.35/master branch has been updated by Szabolcs Nagy <nsz@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=f5f7144dfcbf2a11fd2c17316c213928307c1db3

commit f5f7144dfcbf2a11fd2c17316c213928307c1db3
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Fri Jun 15 16:14:58 2018 +0100

    rtld: Use generic argv adjustment in ld.so [BZ #23293]
    
    When an executable is invoked as
    
      ./ld.so [ld.so-args] ./exe [exe-args]
    
    then the argv is adujusted in ld.so before calling the entry point of
    the executable so ld.so args are not visible to it.  On most targets
    this requires moving argv, env and auxv on the stack to ensure correct
    stack alignment at the entry point.  This had several issues:
    
    - The code for this adjustment on the stack is written in asm as part
      of the target specific ld.so _start code which is hard to maintain.
    
    - The adjustment is done after _dl_start returns, where it's too late
      to update GLRO(dl_auxv), as it is already readonly, so it points to
      memory that was clobbered by the adjustment. This is bug 23293.
    
    - _environ is also wrong in ld.so after the adjustment, but it is
      likely not used after _dl_start returns so this is not user visible.
    
    - _dl_argv was updated, but for this it was moved out of relro, which
      changes security properties across targets unnecessarily.
    
    This patch introduces a generic _dl_start_args_adjust function that
    handles the argument adjustments after ld.so processed its own args
    and before relro protection is applied.
    
    The same algorithm is used on all targets, _dl_skip_args is now 0, so
    existing target specific adjustment code is no longer used.  The bug
    affects aarch64, alpha, arc, arm, csky, ia64, nios2, s390-32 and sparc,
    other targets don't need the change in principle, only for consistency.
    
    The GNU Hurd start code relied on _dl_skip_args after dl_main returned,
    now it checks directly if args were adjusted and fixes the Hurd startup
    data accordingly.
    
    Follow up patches can remove _dl_skip_args and DL_ARGV_NOT_RELRO.
    
    Tested on aarch64-linux-gnu and cross tested on i686-gnu.
    
    Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
    (cherry picked from commit ad43cac44a6860eaefcadadfb2acb349921e96bf)
Comment 11 Sourceware Commits 2022-05-19 09:52:08 UTC
The release/2.35/master branch has been updated by Szabolcs Nagy <nsz@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=2b128a7d30f5f808c5246034f71d249010521f1b

commit 2b128a7d30f5f808c5246034f71d249010521f1b
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Tue May 3 13:18:04 2022 +0100

    linux: Add a getauxval test [BZ #23293]
    
    This is for bug 23293 and it relies on the glibc test system running
    tests via explicit ld.so invokation by default.
    
    Reviewed-by: Florian Weimer <fweimer@redhat.com>
    Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
    (cherry picked from commit 9faf5262c77487c96da8a3e961b88c0b1879e186)
Comment 12 Sourceware Commits 2022-05-19 15:49:44 UTC
The release/2.34/master branch has been updated by Szabolcs Nagy <nsz@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c73c79af7d6f1124fbfa5d935b4f620217d6a2ec

commit c73c79af7d6f1124fbfa5d935b4f620217d6a2ec
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Fri Jun 15 16:14:58 2018 +0100

    rtld: Use generic argv adjustment in ld.so [BZ #23293]
    
    When an executable is invoked as
    
      ./ld.so [ld.so-args] ./exe [exe-args]
    
    then the argv is adujusted in ld.so before calling the entry point of
    the executable so ld.so args are not visible to it.  On most targets
    this requires moving argv, env and auxv on the stack to ensure correct
    stack alignment at the entry point.  This had several issues:
    
    - The code for this adjustment on the stack is written in asm as part
      of the target specific ld.so _start code which is hard to maintain.
    
    - The adjustment is done after _dl_start returns, where it's too late
      to update GLRO(dl_auxv), as it is already readonly, so it points to
      memory that was clobbered by the adjustment. This is bug 23293.
    
    - _environ is also wrong in ld.so after the adjustment, but it is
      likely not used after _dl_start returns so this is not user visible.
    
    - _dl_argv was updated, but for this it was moved out of relro, which
      changes security properties across targets unnecessarily.
    
    This patch introduces a generic _dl_start_args_adjust function that
    handles the argument adjustments after ld.so processed its own args
    and before relro protection is applied.
    
    The same algorithm is used on all targets, _dl_skip_args is now 0, so
    existing target specific adjustment code is no longer used.  The bug
    affects aarch64, alpha, arc, arm, csky, ia64, nios2, s390-32 and sparc,
    other targets don't need the change in principle, only for consistency.
    
    The GNU Hurd start code relied on _dl_skip_args after dl_main returned,
    now it checks directly if args were adjusted and fixes the Hurd startup
    data accordingly.
    
    Follow up patches can remove _dl_skip_args and DL_ARGV_NOT_RELRO.
    
    Tested on aarch64-linux-gnu and cross tested on i686-gnu.
    
    Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
    (cherry picked from commit ad43cac44a6860eaefcadadfb2acb349921e96bf)
Comment 13 Sourceware Commits 2022-05-19 15:49:49 UTC
The release/2.34/master branch has been updated by Szabolcs Nagy <nsz@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=b2585cae2854d7d2868fb2e51e2796042c5e0679

commit b2585cae2854d7d2868fb2e51e2796042c5e0679
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Tue May 3 13:18:04 2022 +0100

    linux: Add a getauxval test [BZ #23293]
    
    This is for bug 23293 and it relies on the glibc test system running
    tests via explicit ld.so invokation by default.
    
    Reviewed-by: Florian Weimer <fweimer@redhat.com>
    Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
    (cherry picked from commit 9faf5262c77487c96da8a3e961b88c0b1879e186)