[PATCH] x86: Optimize atomic_compare_and_exchange_[val|bool]_acq [BZ #28537]
Noah Goldstein
goldstein.w.n@gmail.com
Wed Nov 3 17:25:49 GMT 2021
On Wed, Nov 3, 2021 at 10:05 AM H.J. Lu via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> From the CPU's point of view, getting a cache line for writing is more
> expensive than reading. See Appendix A.2 Spinlock in:
>
> https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/xeon-lock-scaling-analysis-paper.pdf
>
> The full compare and swap will grab the cache line exclusive and cause
> excessive cache line bouncing. Check the current memory value first and
> return immediately if writing cache line may fail to reduce cache line
> bouncing on contended locks.
How does this affect the no-contention case? AFAIK that is the common case
in most systems. This seems like something a user can do if their application
can benefit from it and not something for everyone.
>
> This fixes BZ# 28537.
> ---
> sysdeps/x86/atomic-machine.h | 14 ++++++++++++--
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/sysdeps/x86/atomic-machine.h b/sysdeps/x86/atomic-machine.h
> index 2692d94a92..92c7cf58b7 100644
> --- a/sysdeps/x86/atomic-machine.h
> +++ b/sysdeps/x86/atomic-machine.h
> @@ -73,9 +73,19 @@ typedef uintmax_t uatomic_max_t;
> #define ATOMIC_EXCHANGE_USES_CAS 0
>
> #define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
> - __sync_val_compare_and_swap (mem, oldval, newval)
> + ({ __typeof (*(mem)) oldmem = *(mem), ret; \
> + ret = (oldmem == (oldval) \
> + ? __sync_val_compare_and_swap (mem, oldval, newval) \
> + : oldmem); \
> + ret; })
> #define atomic_compare_and_exchange_bool_acq(mem, newval, oldval) \
> - (! __sync_bool_compare_and_swap (mem, oldval, newval))
> + ({ __typeof (*(mem)) old = *(mem); \
> + int ret; \
> + if (old != (oldval)) \
> + ret = 1; \
> + else \
> + ret = !__sync_bool_compare_and_swap (mem, oldval, newval); \
> + ret; })
>
>
> #define __arch_c_compare_and_exchange_val_8_acq(mem, newval, oldval) \
> --
> 2.33.1
>
More information about the Libc-alpha
mailing list