[PATCH v9 6/9] x86: Add AVX2 optimized chacha20

[Noah Goldstein]

On Wed, Jul 13, 2022 at 12:32 PM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 13/07/22 15:07, Noah Goldstein wrote:
> > On Wed, Jul 13, 2022 at 10:40 AM Adhemerval Zanella via Libc-alpha
> > <libc-alpha@sourceware.org> wrote:
> >>
> >> From: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>
> >> diff --git a/sysdeps/x86_64/chacha20_arch.h b/sysdeps/x86_64/chacha20_arch.h
> >> index 5738c840a9..bfdc6c0a36 100644
> >> --- a/sysdeps/x86_64/chacha20_arch.h
> >> +++ b/sysdeps/x86_64/chacha20_arch.h
> >> @@ -23,16 +23,26 @@
> >>  unsigned int __chacha20_sse2_blocks4 (uint32_t *state, uint8_t *dst,
> >>                                       const uint8_t *src, size_t nblks)
> >>       attribute_hidden;
> >> +unsigned int __chacha20_avx2_blocks8 (uint32_t *state, uint8_t *dst,
> >> +                                     const uint8_t *src, size_t nblks)
> >> +     attribute_hidden;
> >>
> >>  static inline void
> >>  chacha20_crypt (uint32_t *state, uint8_t *dst, const uint8_t *src,
> >>                 size_t bytes)
> >>  {
> >> -  _Static_assert (CHACHA20_BUFSIZE % 4 == 0,
> >> -                 "CHACHA20_BUFSIZE not multiple of 4");
> >> -  _Static_assert (CHACHA20_BUFSIZE >= CHACHA20_BLOCK_SIZE * 4,
> >> -                 "CHACHA20_BUFSIZE <= CHACHA20_BLOCK_SIZE * 4");
> >> +  _Static_assert (CHACHA20_BUFSIZE % 4 == 0 && CHACHA20_BUFSIZE % 8 == 0,
> >> +                 "CHACHA20_BUFSIZE not multiple of 4 or 8");
> >> +  _Static_assert (CHACHA20_BUFSIZE >= CHACHA20_BLOCK_SIZE * 8,
> >> +                 "CHACHA20_BUFSIZE < CHACHA20_BLOCK_SIZE * 8");
> >> +  const struct cpu_features* cpu_features = __get_cpu_features ();
> >>
> >> -  __chacha20_sse2_blocks4 (state, dst, src,
> >> -                          CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
> >> +  /* AVX2 version uses vzeroupper, so disable it if RTM is enabled.  */
> >
> > Since `arc4random ()` might need to read from /dev/urandom I don't
> > think this function could ever truly be RTM safe so we may not care.>
> > If im missing something we do want to support RTM, should there be a
> > '!CPU_FEATURE_USABLE_P (cpu_features, RTM)' check for the avx2
> > implementation?
> >
> I don't fully recall the issue regarding RTM to be sincere (just that
> we had to rework some ifunc selection to handle it).

In this case we don't need to support RTM so no need.
>
> >
> >
> >> +  if (CPU_FEATURE_USABLE_P (cpu_features, AVX2)
> >> +      && !CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER))
> >
> > Can you use the X86_ISA_* macro?
> >
> > In this case the code would be:
> >
> >   if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2)
> >       && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER, !))
>
> Yes it would work. I have changed to the following to support the x86_64 isa
> work you have been doing:

Thanks.
>
> --
> #if MINIMUM_X86_ISA_LEVEL > 2
>   __chacha20_avx2_blocks8 (state, dst, src,
>                            CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
> #else
>   const struct cpu_features* cpu_features = __get_cpu_features ();
>
>   /* AVX2 version uses vzeroupper, so disable it if RTM is enabled.  */
>   if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2)
>       && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER, !))
>     __chacha20_avx2_blocks8 (state, dst, src,
>                              CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
>   else
>     __chacha20_sse2_blocks4 (state, dst, src,
>                              CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
> #endif
> --
>
> I am aware that X86_ISA_CPU_FEATURE_USABLE_P will const-eval to 1 if the
> ISA is higher enough, but I think the code is slight clear (specially
> when the reader is not aware of the const-eval).
>

Think this is good. Thanks and sorry for the last minute requests.
>
> >
> >
> >> +    __chacha20_avx2_blocks8 (state, dst, src,
> >> +                            CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
> >> +  else
> >> +    __chacha20_sse2_blocks4 (state, dst, src,
> >> +                            CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
> >>  }
> >> --
> >> 2.34.1
> >>