A collection of LD_AUDIT bugs that are important for tools (with better formatting for this list)

[Adhemerval Zanella]

On 23/06/2021 14:42, Ben Woodard wrote:
> 
> 
>> On Jun 17, 2021, at 4:06 PM, Adhemerval Zanella <adhemerval.zanella@linaro.org> wrote:
>>
>>
>>
>> On 17/06/2021 17:09, Florian Weimer wrote:
>>> * Adhemerval Zanella:
>>>
>>>>
>>>>
>>>> * SVE support: as indicated by Szabolcs SVE calls are marked with the 
>>>>  STO_AARCH64_VARIANT_PCS and thus explicit not supported by dynamic loader. 
> 
> To me this sounds like partly a toolchain issue. The aarch64 PCS does define the ABI for SVE calls. I haven’t checked what GCC/binutils does in quite a while. It seems like the  STO_AARCH64_VARIANT_PCS was an expedient for SVE when it first came out and the semantics that it defines where all the registers are caller preserved makes it very difficult to implement around.
> 
> For LAV_CURRENT=2 what I planned to do was:

Now that I have finished the various audit issues that John Mellor-Crummey 
has brought up, I think I have a better idea of how to address it.

> 
> diff --git a/sysdeps/aarch64/bits/link.h b/sysdeps/aarch64/bits/link.h
> index ca76087ee1..390b12a826 100644
> --- a/sysdeps/aarch64/bits/link.h
> +++ b/sysdeps/aarch64/bits/link.h
> @@ -20,13 +20,24 @@
>  # error "Never include <bits/link.h> directly; use <link.h> instead."
>  #endif
>  
> +typedef struct La_sve_regs {
> +  uint16_t    *lr_preg[3];
> +  __uint128_t *lr_zreg[8];
> +} La_sve_regs;
> +
>  /* Registers for entry into PLT on AArch64.  */
>  typedef struct La_aarch64_regs
>  {
>    uint64_t    lr_xreg[9];
> -  __uint128_t lr_vreg[8];
>    uint64_t    lr_sp;
>    uint64_t    lr_lr;
> +  char    lr_sve; /* 0 - no SVE,
> +                        1-16 length of the SVE registers in vq (128 bits) */
> +  union {
> +    /* when sve!=0 accessing the lr_vreg is undefined */
> +    __uint128_t lr_vreg[8];
> +    La_sve_regs lr_zreg;
> +  };
>  } La_aarch64_regs;
>  
>  /* Return values for calls from PLT on AArch64.  */
> @@ -34,9 +45,14 @@ typedef struct La_aarch64_retval
>  {
>    /* Up to eight integer registers can be used for a return value.  */
>    uint64_t    lrv_xreg[8];
> -  /* Up to eight V registers can be used for a return value.  */
> -  __uint128_t lrv_vreg[8];
> -
> +  char        lrv_sve; /* 0 - no SVE,
> +                         1-16 length of the SVE registers in vq (128 bits) */
> +  union{
> +    /* Up to eight V registers can be used for a return value.
> +       When sve!=0 accessing the lr_vreg is undefined */
> +    __uint128_t lrv_vreg[8];
> +    La_sve_regs lrv_zreg;
> +  };
>  } La_aarch64_retval;
>  __BEGIN_DECLS

My idea is to do something similar:
---
typedef struct La_sve_regs
{
  uint16_t    *lr_preg;
  long double *lr_zreg;
} La_sve_regs;

typedef struct La_aarch64_regs
{
  [...]
  uint8_t  lr_sve;           /* 0 - no SVE
                                1-16 length of the SVE registers in vq (128 bits)  */
  La_sve_regs *lr_sve_regs;  /* NULL - no SVE.  */
};

typedef struct La_aarch64_retval
{
  [...]
  uint8_t  lr_sve;           /* 0 - no SVE
                                1-16 length of the SVE registers in vq (128 bits)  */
  La_sve_regs *lr_sve_regs;  /* NULL - no SVE.  */
}
---

The _dl_runtime_resolve will be responsible to allocate on the stack the required
space for the La_sve_regs and setup the La_aarch64_regs and La_aarch64_retval internal
pointers.  It has the advantage of allocate only the required space and if the we 
can distinguish if the symbol does use SVE we can avoid the performance issue for 
symbols that do not use SVE. The downside is it would require a potential large stack 
space.

>  
> However, that would require toolchain support and another hint in st_other which separates SVE calls from other uses of STO_AARCH64_VARIANT_PCS like STO_AARCH64_VARIANT_SVE. Then the runtime linker could populate the lrv_sve with information from the lrv_vreg with the size of the vector registers from the processor’s registers. 

I think it should be feasible to assume now that STO_AARCH64_VARIANT_PCS 
means SVE, which meant that we can use it _dl_runtime_resolve to skip
the save/restore if the symbol follows the default standard procedure
call.  

> 
> There are at least two problems with that approach. 
> 1) who allocates the lr_zreg pointers in the la_sve_regs and how long should they be? Do they always have to be allocated to be the max size 2048 bits?
> 2) I hadn’t worked out how to handle functions that return things in the SVE regs. Do we need two new flags in st_other? STO_AARCH64_VARIANT_SVE and STO_AARCH64_VARIANT_SVERET?

I don't think we need the two extra flags to handle SVE calls, but I am
also assuming that STO_AARCH64_VARIANT_PCS will be used solely for SVE.
And it has the extra problem of using two extra flags is not backward
compatible.

I presume that if ARM wants to push for another procedure call variant
on linux-gnu I would expect another flag.

> 
> Then there was the question in the future could there be: big.LITTLE processors where some big processors had SVE registers of one length while the LITTLE processors had different ones? 

Although I find this kind of setup unlikely, my expectation is either that it
would be transparent to userland (either kernel will emulate the required 
instructions or it will bind process with STO_AARCH64_VARIANT_PCS to cores 
with SVE).