[PATCH] AArch64: Improve memset when len is 64

[Andrew Pinski]

On Mon, Mar 2, 2026 at 9:49 AM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
>
> Change the mask to 48 to support len==64. The second memory store now accesses
> offset 32, whereas the third one accesses offset 16.  As a result performance
> for len==64 almost doubles.
>
> Passes regress, OK for commit?

LGTM.

Thanks,
Andrew

>
> ---
>
> diff --git a/sysdeps/aarch64/memset.S b/sysdeps/aarch64/memset.S
> index ff59fbb6a26f2895558390618f3a3003b1d0835e..b62b536d6a211cc1aac6f9fcc33ee8385cdcee35 100644
> --- a/sysdeps/aarch64/memset.S
> +++ b/sysdeps/aarch64/memset.S
> @@ -45,10 +45,10 @@ ENTRY (MEMSET)
>
>         add     dstend, dstin, count
>         cmp     count, 64
> -       b.hs    L(set_128)
> +       b.hi    L(set_128)
>
> -       /* Set 16..63 bytes.  */
> -       mov     off, 16
> +       /* Set 16..64 bytes.  */
> +       mov     off, 48
>         and     off, off, count, lsr 1
>         sub     dstend2, dstend, off
>         str     q0, [dstin]
> diff --git a/sysdeps/aarch64/multiarch/memset_sve_zva64.S b/sysdeps/aarch64/multiarch/memset_sve_zva64.S
> index c593aae3bf9b0f7cd9a6b52c23db960ef0014da6..654e6d47e03f297e61aa871eb34489f6dcd14972 100644
> --- a/sysdeps/aarch64/multiarch/memset_sve_zva64.S
> +++ b/sysdeps/aarch64/multiarch/memset_sve_zva64.S
> @@ -45,10 +45,10 @@ ENTRY (__memset_sve_zva64)
>
>         add     dstend, dstin, count
>         cmp     count, 64
> -       b.hs    L(set_128)
> +       b.hi    L(set_128)
>
> -       /* Set 16..63 bytes.  */
> -       mov     off, 16
> +       /* Set 16..64 bytes.  */
> +       mov     off, 48
>         and     off, off, count, lsr 1
>         sub     dstend2, dstend, off
>         str     q0, [dstin]
>