[PATCH v9 6/9] x86: Add AVX2 optimized chacha20

[Adhemerval Zanella Netto]

On 13/07/22 15:07, Noah Goldstein wrote:
> On Wed, Jul 13, 2022 at 10:40 AM Adhemerval Zanella via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
>>
>> From: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>
>> diff --git a/sysdeps/x86_64/chacha20_arch.h b/sysdeps/x86_64/chacha20_arch.h
>> index 5738c840a9..bfdc6c0a36 100644
>> --- a/sysdeps/x86_64/chacha20_arch.h
>> +++ b/sysdeps/x86_64/chacha20_arch.h
>> @@ -23,16 +23,26 @@
>>  unsigned int __chacha20_sse2_blocks4 (uint32_t *state, uint8_t *dst,
>>                                       const uint8_t *src, size_t nblks)
>>       attribute_hidden;
>> +unsigned int __chacha20_avx2_blocks8 (uint32_t *state, uint8_t *dst,
>> +                                     const uint8_t *src, size_t nblks)
>> +     attribute_hidden;
>>
>>  static inline void
>>  chacha20_crypt (uint32_t *state, uint8_t *dst, const uint8_t *src,
>>                 size_t bytes)
>>  {
>> -  _Static_assert (CHACHA20_BUFSIZE % 4 == 0,
>> -                 "CHACHA20_BUFSIZE not multiple of 4");
>> -  _Static_assert (CHACHA20_BUFSIZE >= CHACHA20_BLOCK_SIZE * 4,
>> -                 "CHACHA20_BUFSIZE <= CHACHA20_BLOCK_SIZE * 4");
>> +  _Static_assert (CHACHA20_BUFSIZE % 4 == 0 && CHACHA20_BUFSIZE % 8 == 0,
>> +                 "CHACHA20_BUFSIZE not multiple of 4 or 8");
>> +  _Static_assert (CHACHA20_BUFSIZE >= CHACHA20_BLOCK_SIZE * 8,
>> +                 "CHACHA20_BUFSIZE < CHACHA20_BLOCK_SIZE * 8");
>> +  const struct cpu_features* cpu_features = __get_cpu_features ();
>>
>> -  __chacha20_sse2_blocks4 (state, dst, src,
>> -                          CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
>> +  /* AVX2 version uses vzeroupper, so disable it if RTM is enabled.  */
> 
> Since `arc4random ()` might need to read from /dev/urandom I don't
> think this function could ever truly be RTM safe so we may not care.> 
> If im missing something we do want to support RTM, should there be a
> '!CPU_FEATURE_USABLE_P (cpu_features, RTM)' check for the avx2
> implementation?
> 
I don't fully recall the issue regarding RTM to be sincere (just that
we had to rework some ifunc selection to handle it).

> 
> 
>> +  if (CPU_FEATURE_USABLE_P (cpu_features, AVX2)
>> +      && !CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER))
> 
> Can you use the X86_ISA_* macro?
> 
> In this case the code would be:
> 
>   if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2)
>       && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER, !))

Yes it would work. I have changed to the following to support the x86_64 isa
work you have been doing:

--
#if MINIMUM_X86_ISA_LEVEL > 2
  __chacha20_avx2_blocks8 (state, dst, src,
                           CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
#else
  const struct cpu_features* cpu_features = __get_cpu_features ();

  /* AVX2 version uses vzeroupper, so disable it if RTM is enabled.  */
  if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2)
      && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, Prefer_No_VZEROUPPER, !))
    __chacha20_avx2_blocks8 (state, dst, src,
                             CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
  else
    __chacha20_sse2_blocks4 (state, dst, src,
                             CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
#endif
--

I am aware that X86_ISA_CPU_FEATURE_USABLE_P will const-eval to 1 if the
ISA is higher enough, but I think the code is slight clear (specially
when the reader is not aware of the const-eval). 


> 
> 
>> +    __chacha20_avx2_blocks8 (state, dst, src,
>> +                            CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
>> +  else
>> +    __chacha20_sse2_blocks4 (state, dst, src,
>> +                            CHACHA20_BUFSIZE / CHACHA20_BLOCK_SIZE);
>>  }
>> --
>> 2.34.1
>>