This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [RESEND PATCH v9] Mutex: Add pthread mutex tunables
+Carlos
On 2018/11/27 上午10:43, kemi wrote:
> PING~
>
> On 2018/11/22 上午9:45, Kemi Wang wrote:
>> This patch does not have any functionality change, we only provide a spin
>> count tunes for pthread adaptive spin mutex. The tunable
>> glibc.pthread.mutex_spin_count tunes can be used by system administrator to
>> squeeze system performance according to different hardware capabilities and
>> workload characteristics.
>>
>> The maximum value of spin count is limited to 30000 to avoid the overflow
>> of mutex->__data.__spins variable with the possible type of short in
>> pthread_mutex_lock ().
>>
>> The default value of spin count is set to 100 with the reference to the
>> previous number of times of spinning via trylock. This value would be
>> architecture-specific and can be tuned with kinds of benchmarks to fit most
>> cases in future.
>>
>> * sysdeps/nptl/dl-tunables.list: Add glibc.pthread.mutex_spin_count entry.
>> * manual/tunables.texi: Add glibc.pthread.mutex_spin_count description.
>> * nptl/pthread_mutex_conf.h: New file.
>> * nptl/pthread_mutex_conf.c: New file.
>> * nptl/Makefile: Add pthread_mutex_conf.c for compilation.
>> * nptl/nptl-init.c: Put pthread mutex tunable initialization in pthread
>> initialization.
>> * nptl/pthreadP.h: Refine MAX_ADAPTIVE_COUNT macro definition
>>
>> I would extend my appreciation sincerely to H.J.Lu for his help to refine
>> this patch series.
>>
>> ChangeLog:
>> V8->V9:
>> a) Add the "unistd.h" header file back, or it will cause build
>> regression on 32 bits system.
>> b) Rebase on the latest master branch
>> c) Tested on x86_64 with build-many-glibcs.py
>>
>> V7->V8:
>> a) Refine the pthread tunables description in manual/tunables.texi
>> accordingly to Carlos O'Donell and Rical Jason.
>>
>> V6->V7:
>> a) Patch is refined by H.J.Lu
>>
>> V5->V6:
>> a) Missing "pthread mutex tunables" entry in the menu of tunables.texi,
>> add it.
>>
>> V4->V5
>> a) Put mutex tunable (glibc.mutex.spin_count) initialization as part of
>> overall pthread initialization, that would avoid the extra relocation,
>> as suggested by Florian Weimer. Thanks for pointing it out!
>> b) Move the READ_ONLY_SPIN macro definition from the third patch to
>> this patch
>>
>> V3->V4
>> a) Add comments in elf/dl-tunables.list
>>
>> V2->V3
>> a) Polish the description of glibc.mutex.spin_count tunable with the
>> help from Rical Jasan.
>> b) Get rid of the TUNABLE_CALLBACK_FNDECL macros in
>> pthread_mutex_conf.c, as suggested by Florian Weimer.
>> c) Adjust the default value of spin count to 100 with the reference of
>> the previous spinning way via trylock.
>>
>> V1->V2
>> a) Renamed nptl/mutex-conf.h -> nptl/pthread_mutex_conf.h
>> b) Renamed nptl/mutex-conf.c -> nptl/pthread_mutex_conf.c
>> c) Change the Makefile to compile pthread_mutex_conf.c
>> d) Modify the copyright "2013-2018" -> "2018" for new added files
>> e) Fix the indentation issue (tab -> double space) in
>> elf/dl-tunables.list
>> f) Remove the env alias LD_SPIN_COUNT in elf/dl-tunables.list
>> g) Fix the typo errors and refresh glibc.mutex.spin_count tunable
>> description in manual/tunables.texi.
>> h) Fix the indentation issue in nptl/pthread_mutex_conf.c
>> i) Fix the indentation issue for nested preprocessor (add one space for
>> each level)
>>
>> Suggested-by: Andi Kleen <andi.kleen@intel.com>
>> Signed-off-by: Kemi.wang <kemi.wang@intel.com>
>> ---
>> manual/tunables.texi | 27 +++++++++++++++++++++++
>> nptl/Makefile | 2 +-
>> nptl/nptl-init.c | 5 +++++
>> nptl/pthreadP.h | 24 +++++++++++++++-----
>> nptl/pthread_mutex_conf.c | 27 +++++++++++++++++++++++
>> nptl/pthread_mutex_conf.h | 45 ++++++++++++++++++++++++++++++++++++++
>> sysdeps/nptl/dl-tunables.list | 51 +++++++++++++++++++++++++++++++++++++++++++
>> 7 files changed, 174 insertions(+), 7 deletions(-)
>> create mode 100644 nptl/pthread_mutex_conf.c
>> create mode 100644 nptl/pthread_mutex_conf.h
>> create mode 100644 sysdeps/nptl/dl-tunables.list
>>
>> diff --git a/manual/tunables.texi b/manual/tunables.texi
>> index 3345a23..33fbe1e 100644
>> --- a/manual/tunables.texi
>> +++ b/manual/tunables.texi
>> @@ -32,6 +32,7 @@ their own namespace.
>> * Tunable names:: The structure of a tunable name
>> * Memory Allocation Tunables:: Tunables in the memory allocation subsystem
>> * Elision Tunables:: Tunables in elision subsystem
>> +* POSIX Thread Tunables:: Tunables in the POSIX thread subsystem
>> * Hardware Capability Tunables:: Tunables that modify the hardware
>> capabilities seen by @theglibc{}
>> @end menu
>> @@ -281,6 +282,32 @@ of try lock attempts.
>> The default value of this tunable is @samp{3}.
>> @end deftp
>>
>> +@node POSIX Thread Tunables
>> +@section POSIX Thread Tunables
>> +@cindex pthread mutex tunables
>> +@cindex thread mutex tunables
>> +@cindex mutex tunables
>> +@cindex tunables thread mutex
>> +
>> +@deftp {Tunable namespace} glibc.pthread
>> +The behavior of POSIX thread can be tuned to gain performance improvements
>> +according to specific hardware capabilities and workload characteristics by
>> +setting the following tunables in the @code{pthread} namespace:
>> +@end deftp
>> +
>> +@deftp Tunable glibc.pthread.mutex_spin_count
>> +The @code{glibc.pthread.mutex_spin_count} tunable sets the maximum number of times
>> +a thread should spin on the lock before calling into the kernel to block.
>> +Adaptive spin is used for mutexes initialized with the
>> +@code{PTHREAD_MUTEX_ADAPTIVE_NP} GNU extension. It affects both
>> +@code{pthread_mutex_lock} and @code{pthread_mutex_timedlock}.
>> +
>> +The spinning is done until either the maximum spin count is reached or
>> +the lock is acquired.
>> +
>> +The default value of this tunable is @samp{100}.
>> +@end deftp
>> +
>> @node Hardware Capability Tunables
>> @section Hardware Capability Tunables
>> @cindex hardware capability tunables
>> diff --git a/nptl/Makefile b/nptl/Makefile
>> index 49b6faa..284f815 100644
>> --- a/nptl/Makefile
>> +++ b/nptl/Makefile
>> @@ -145,7 +145,7 @@ libpthread-routines = nptl-init nptlfreeres vars events version pt-interp \
>> mtx_destroy mtx_init mtx_lock mtx_timedlock \
>> mtx_trylock mtx_unlock call_once cnd_broadcast \
>> cnd_destroy cnd_init cnd_signal cnd_timedwait cnd_wait \
>> - tss_create tss_delete tss_get tss_set
>> + tss_create tss_delete tss_get tss_set pthread_mutex_conf
>> # pthread_setuid pthread_seteuid pthread_setreuid \
>> # pthread_setresuid \
>> # pthread_setgid pthread_setegid pthread_setregid \
>> diff --git a/nptl/nptl-init.c b/nptl/nptl-init.c
>> index 907411d..adf99f1 100644
>> --- a/nptl/nptl-init.c
>> +++ b/nptl/nptl-init.c
>> @@ -38,6 +38,7 @@
>> #include <kernel-features.h>
>> #include <libc-pointer-arith.h>
>> #include <pthread-pids.h>
>> +#include <pthread_mutex_conf.h>
>>
>> #ifndef TLS_MULTIPLE_THREADS_IN_TCB
>> /* Pointer to the corresponding variable in libc. */
>> @@ -431,6 +432,10 @@ __pthread_initialize_minimal_internal (void)
>>
>> /* Determine whether the machine is SMP or not. */
>> __is_smp = is_smp_system ();
>> +
>> +#if HAVE_TUNABLES
>> + pthread_tunables_init ();
>> +#endif
>> }
>> strong_alias (__pthread_initialize_minimal_internal,
>> __pthread_initialize_minimal)
>> diff --git a/nptl/pthreadP.h b/nptl/pthreadP.h
>> index 19efe1e..9bae78e 100644
>> --- a/nptl/pthreadP.h
>> +++ b/nptl/pthreadP.h
>> @@ -47,12 +47,6 @@
>> #endif
>>
>>
>> -/* Adaptive mutex definitions. */
>> -#ifndef MAX_ADAPTIVE_COUNT
>> -# define MAX_ADAPTIVE_COUNT 100
>> -#endif
>> -
>> -
>> /* Magic cookie representing robust mutex with dead owner. */
>> #define PTHREAD_MUTEX_INCONSISTENT INT_MAX
>> /* Magic cookie representing not recoverable robust mutex. */
>> @@ -188,6 +182,24 @@ enum
>> #define __PTHREAD_COND_SHARED_MASK 1
>>
>>
>> +#if HAVE_TUNABLES
>> +struct mutex_config
>> +{
>> + int spin_count;
>> +};
>> +
>> +extern struct mutex_config __mutex_aconf;
>> +#endif
>> +
>> +/* Adaptive mutex definitions. */
>> +#ifndef MAX_ADAPTIVE_COUNT
>> +# if HAVE_TUNABLES
>> +# define MAX_ADAPTIVE_COUNT __mutex_aconf.spin_count
>> +# else
>> +# define MAX_ADAPTIVE_COUNT 100
>> +# endif
>> +#endif
>> +
>> /* Internal variables. */
>>
>>
>> diff --git a/nptl/pthread_mutex_conf.c b/nptl/pthread_mutex_conf.c
>> new file mode 100644
>> index 0000000..0da4afe
>> --- /dev/null
>> +++ b/nptl/pthread_mutex_conf.c
>> @@ -0,0 +1,27 @@
>> +/* pthread_mutex_conf.c: Pthread mutex tunable parameters.
>> + Copyright (C) 2018 Free Software Foundation, Inc.
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library; if not, see
>> + <http://www.gnu.org/licenses/>. */
>> +
>> +#if HAVE_TUNABLES
>> +#include <pthreadP.h>
>> +struct mutex_config __mutex_aconf =
>> +{
>> + /* The maximum number of times a thread should spin on the lock before
>> + calling into kernel to block. */
>> + .spin_count = 100,
>> +};
>> +#endif
>> diff --git a/nptl/pthread_mutex_conf.h b/nptl/pthread_mutex_conf.h
>> new file mode 100644
>> index 0000000..97ff98e
>> --- /dev/null
>> +++ b/nptl/pthread_mutex_conf.h
>> @@ -0,0 +1,45 @@
>> +/* pthread_mutex_conf.h: Pthread mutex tunable parameters.
>> + Copyright (C) 2018 Free Software Foundation, Inc.
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library; if not, see
>> + <http://www.gnu.org/licenses/>. */
>> +#ifndef _PTHREAD_MUTEX_CONF_H
>> +#define _PTHREAD_MUTEX_CONF_H 1
>> +
>> +#if HAVE_TUNABLES
>> +# define TUNABLE_NAMESPACE pthread
>> +# include <stdbool.h>
>> +# include <stdint.h>
>> +# include <pthreadP.h>
>> +# include <pthread_mutex_conf.h>
>> +# include <unistd.h> /* Get STDOUT_FILENO for _dl_printf. */
>> +# include <elf/dl-tunables.h>
>> +
>> +extern struct mutex_config __mutex_aconf attribute_hidden;
>> +
>> +static void
>> +TUNABLE_CALLBACK (set_mutex_spin_count) (tunable_val_t *valp)
>> +{
>> + __mutex_aconf.spin_count = (int32_t) (valp)->numval;
>> +}
>> +
>> +static void pthread_tunables_init (void)
>> +{
>> + TUNABLE_GET (mutex_spin_count, int32_t,
>> + TUNABLE_CALLBACK (set_mutex_spin_count));
>> +}
>> +#endif
>> +
>> +#endif
>> diff --git a/sysdeps/nptl/dl-tunables.list b/sysdeps/nptl/dl-tunables.list
>> new file mode 100644
>> index 0000000..1cb52e8
>> --- /dev/null
>> +++ b/sysdeps/nptl/dl-tunables.list
>> @@ -0,0 +1,51 @@
>> +# Copyright (C) 2018 Free Software Foundation, Inc.
>> +# This file is part of the GNU C Library.
>> +
>> +# The GNU C Library is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU Lesser General Public
>> +# License as published by the Free Software Foundation; either
>> +# version 2.1 of the License, or (at your option) any later version.
>> +
>> +# The GNU C Library is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> +# Lesser General Public License for more details.
>> +
>> +# You should have received a copy of the GNU Lesser General Public
>> +# License along with the GNU C Library; if not, see
>> +# <http://www.gnu.org/licenses/>.
>> +
>> +# Allowed attributes for tunables:
>> +#
>> +# type: Defaults to STRING
>> +# minval: Optional minimum acceptable value
>> +# maxval: Optional maximum acceptable value
>> +# env_alias: An alias environment variable
>> +# security_level: Specify security level of the tunable. Valid values are:
>> +#
>> +# SXID_ERASE: (default) Don't read for AT_SECURE binaries and
>> +# removed so that child processes can't read it.
>> +# SXID_IGNORE: Don't read for AT_SECURE binaries, but retained for
>> +# non-AT_SECURE subprocesses.
>> +# NONE: Read all the time.
>> +
>> +
>> +# The maximum value of spin count is limited to 30000 to avoid the overflow
>> +# of mutex->__data.__spins variable with the possible type of short in
>> +# pthread_mutex_lock ().
>> +#
>> +# The default value of spin count is set to 100 with the reference to the
>> +# previous number of times of spinning via trylock. This value would be
>> +# architecture-specific and can be tuned with kinds of benchmarks to fit
>> +# most cases in future.
>> +
>> +glibc {
>> + pthread {
>> + mutex_spin_count {
>> + type: INT_32
>> + minval: 0
>> + maxval: 30000
>> + default: 100
>> + }
>> + }
>> +}
>>