[Bug nptl/33704] trylock throughput reduces significantly under high concurency load.

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=33704

--- Comment #8 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to Sunil Pandey from comment #7)
> Created attachment 16511 [details]
> pthread_mutex_trylock_patch
> 
> We are interested about pthread_mutex_trylock case PTHREAD_MUTEX_TIMED_NP.

Which is also the same case as PTHREAD_MUTEX_NORMAL since they are the same
type in glibc.

Where you are making the changes looks good to me, but we have a question of
the standard to resolve.

Firstly POSIX says "Unless explicitly stated otherwise, if one of the above
functions returns an error, it is unspecified whether the invocation causes
memory to be synchronized."

However, it also says "The pthread_mutex_trylock() function shall be equivalent
to pthread_mutex_lock(), except that if the mutex object referenced by mutex is
currently locked (by any thread, including the current thread), the call shall
return immediately."

So to operate as-if pthread_mutex_lock(), we need to use an acquire barrier to
load the lock, but if the result is a failure the barrier used could have been
relaxed, and thus there is a conflict here.

It should be OK to relax the ordering from acquire to relaxed for the load of
the lock word where pthread_mutex_trylock() would fail, but then we're not
exactly operating as-if pthread_mutex_lock() and that can have an observable
effect.

Thus there is a chance that some applications have come to expect that
pthread_mutex_trylock() operates as an acquire barrier since the CAS on the
lock word has acquire semantics. I think this would be an incorrect reading of
POSIX, but could still cause backwards compatibility issues that are unclear to
me since we don't have enough data on the exact uses of pthread_mutex_trylock()
in all applications.

I'm OK with relaxing pthread_mutex_trylock to:

 * Relaxed MO load for failed case. (unspecified memory synchornization under
posix)
 * Acquire MO load for successful case. (expected currently)

Thoughts?

> Attached patch improves throughput across board on x86. Following are
> skylake server data.
> 
> Core count: 80
> 
> tcount baseline	pthread_mutex_trylock speedup
> 80     232112   3119301               13.44x
> 64     247737   937424                3.78x
> 32     441441   888738                2.01x
> 16     863201   1529987               1.77x
> 8      1587950  2775892               1.75x
> 4      3208012  6487683               2.02x
> 2      7729917  11811184              1.53x

That looks good, and likely in line with going from acquire MO to relaxed MO.

This needs a microbenchmark and more comments, but is heading in the right
direction.

-- 
You are receiving this mail because:
You are on the CC list for the bug.