[PATCH] x86: Optimize atomic_compare_and_exchange_[val|bool]_acq [BZ #28537]

[Oleh Derevenko]

Arjan

> What the patch does is check non-atomic first if the actual atomic operation has
a chance of working. if it has a chance, the actual normal atomic
operation is done as
before. But if non-atomic read already tells you the cmpxchg has no
chance to succeed, it errors
out early.

The idea of atomic function is that they are intended to work fairly
with any type of memory. In your case, the speculative reads for a
cached device memory may result in cache access only and will prevent
fetching memory updates from the device, thus making the reading
thread "see" the change later than it could.
If you want to make a "RAM-specific" version of compare-n-exchange
give it a distinct specific name.

On Wed, Nov 3, 2021 at 7:00 PM Arjan van de Ven <arjan@linux.intel.com> wrote:
>
> On 11/3/2021 8:50 AM, Oleh Derevenko wrote:
> > Hi, H.J. Lu
> >
> > You may not perform plain reads on values you want to be atomic. This
> > results in undefined behavior.
>
> so the way the patch works is that it does not DEPEND on that read to be atomic.
>
> What the patch does is check non-atomic first if the actual atomic operation has
> a chance of working. if it has a chance, the actual normal atomic operation is done as
> before. But if non-atomic read already tells you the cmpxchg has no chance to succeed, it errors
> out early.
>
> The big gain is for the contended lock case (_acq suffix!). If there's, say, 4 threads spinning
> on a lock. Before this patch these 4 cpu cores would be taking turns bouncing the cacheline around
> super aggressively.. which causes system degradation and worse, also makes the core that will
> eventually unlock the lock wait for the cacheline.
>
> Now with the patch, the "it is locked already" is noticed before the cacheline gets taken exclusive,
> so all 4 spinning cores have the same cacheline in a shared state -- no pingponging.
> Now the core that is going to unlock the lock can now do the exclusive acquire of the cacheline
> without having to fight with those 4 cores in the exclusive acquire fight.
>
>
> > For example, the compiler IS NOT obliged to perform the read with a
> > single CPU instruction -- of course it will not, but it is allowed to
> > read it in two halves and compare them separately. Or it may reuse
> > cached value from previous evaluations.
>
> > This is only the compiler level issue. Similar issues will arise at
> > CPU level with all the kind of memory coherency, caching and
> > instruction reordering.
>
> the cpu in this case won't, the x86 memory model won't allow that
> (and this is in the x86 implementation code)
>
> > Or if the value would cross a cache line boundary the plain read might
> > return half-updated value with the part from one cache line being new
> > and the other part being old.
>
> (I can't say in polite company what cmpxchg across cache lines does)



-- 

Oleh Derevenko

-- Skype with underscore