This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
[ping2][PATCH v2 1/2] Add framework for tunables
- From: Siddhesh Poyarekar <sid at reserved-bit dot com>
- To: libc-alpha at sourceware dot org
- Cc: roland at hack dot frob dot com, carlos at redhat dot com, Andi Kleen <andi at firstfloor dot org>, tuliom at linux dot vnet dot ibm dot com, "Paul E. Murphy" <murphyp at linux dot vnet dot ibm dot com>, munroesj at linux dot vnet dot ibm dot com
- Date: Mon, 1 Feb 2016 15:10:32 +0530
- Subject: [ping2][PATCH v2 1/2] Add framework for tunables
- Authentication-results: sourceware.org; auth=none
- References: <20160116185545 dot GA17772 at devel dot intra dot reserved-bit dot com> <20160125032505 dot GA22776 at devel dot intra dot reserved-bit dot com>
Ping!
On Mon, Jan 25, 2016 at 08:55:06AM +0530, Siddhesh Poyarekar wrote:
> Ping!
>
> On Sun, Jan 17, 2016 at 12:25:50AM +0530, Siddhesh Poyarekar wrote:
> > The tunables framework allows us to uniformly manage and expose global
> > variables inside glibc as switches to users. tunables/README has
> > instructions for glibc developers to add new tunables.
> >
> > Tunables support can be enabled by passing the --enable-tunables
> > configure flag to the configure script. This patch only adds a
> > framework and does not pose any limitations on how tunable values are
> > read from the user. It also adds environment variables used in malloc
> > behaviour tweaking to the tunables framework as a PoC of the
> > compatibility interface.
> >
> > * manual/install.texi: Add --enable-tunables option.
> > * INSTALL: Regenerate.
> > * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
> > (all-subdirs): Add tunables.
> > * config.h.in: Add BUILD_TUNABLES.
> > * config.make.in: Add build-tunables.
> > * configure.ac: Add --enable-tunables option.
> > * configure: Regenerate.
> > * csu/init-first.c: Include tunables.h.
> > (_init): Initialize tunables.
> > * nptl/nptl-init.c [BUILD_TUNABLES]: Include tunables.h.
> > (__pthread_initialize_minimal_internal): Accept argc, argv and
> > envp. Initialize tunables.
> > * csu/libc-start.c (LIBC_START_MAIN): Adjust.
> > * malloc/arena.c [BUILD_TUNABLES]: Include tunables.h. Define
> > TUNABLE_NAMESPACE.
> > (tunable_set_mallopt_top_pad, tunable_set_mallopt_perturb,
> > tunable_set_mallopt_mmap_max, tunable_set_mallopt_arena_max,
> > tunable_set_mallopt_arena_test,
> > tunable_set_mallopt_trim_threshold,
> > tunable_set_mallopt_mmap_threshold,
> > tunable_set_mallopt_check): New functions.
> > (ptmalloc_init): Use them and register tunables.
> > * scripts/gen-tunables.awk: New file.
> > * tunables/Makefile: New file.
> > * tunables/README: New file.
> > * tunables/Versions: New file.
> > * tunables/tunable-list.h: New auto-generated file.
> > * tunables/tunables.c: New file.
> > * tunables/tunables.h: New file.
> > * tunables/tunables.list: New file.
> > ---
> > INSTALL | 6 ++
> > Makeconfig | 9 +++
> > config.h.in | 3 +
> > config.make.in | 1 +
> > configure | 16 +++++
> > configure.ac | 10 +++
> > csu/init-first.c | 7 ++
> > csu/libc-start.c | 4 +-
> > malloc/arena.c | 86 +++++++++++++++++++++++++
> > manual/install.texi | 5 ++
> > nptl/nptl-init.c | 9 ++-
> > scripts/gen-tunables.awk | 84 ++++++++++++++++++++++++
> > tunables/Makefile | 33 ++++++++++
> > tunables/README | 93 +++++++++++++++++++++++++++
> > tunables/Versions | 8 +++
> > tunables/tunable-list.h | 32 ++++++++++
> > tunables/tunables.c | 163 +++++++++++++++++++++++++++++++++++++++++++++++
> > tunables/tunables.h | 110 ++++++++++++++++++++++++++++++++
> > tunables/tunables.list | 12 ++++
> > 19 files changed, 688 insertions(+), 3 deletions(-)
> > create mode 100644 scripts/gen-tunables.awk
> > create mode 100644 tunables/Makefile
> > create mode 100644 tunables/README
> > create mode 100644 tunables/Versions
> > create mode 100644 tunables/tunable-list.h
> > create mode 100644 tunables/tunables.c
> > create mode 100644 tunables/tunables.h
> > create mode 100644 tunables/tunables.list
> >
> > diff --git a/INSTALL b/INSTALL
> > index c70ea9f..dd06e2e 100644
> > --- a/INSTALL
> > +++ b/INSTALL
> > @@ -164,6 +164,12 @@ will be used, and CFLAGS sets optimization options for the compiler.
> > By default for x86_64, the GNU C Library is built with vector math
> > library. Use this option to disable vector math library.
> >
> > +'--enable-tunables'
> > + Tunables support allows additional library parameters to be
> > + customized at runtime for each application. This is an
> > + experimental feature and affects startup time and is thus disabled
> > + by default.
> > +
> > '--build=BUILD-SYSTEM'
> > '--host=HOST-SYSTEM'
> > These options are for cross-compiling. If you specify both options
> > diff --git a/Makeconfig b/Makeconfig
> > index 87a22e8..8ac199b 100644
> > --- a/Makeconfig
> > +++ b/Makeconfig
> > @@ -883,6 +883,11 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \
> > $(foreach lib,$(libof-$(basename $(@F))) \
> > $(libof-$(<F)) $(libof-$(@F)),$(CPPFLAGS-$(lib))) \
> > $(CPPFLAGS-$(<F)) $(CPPFLAGS-$(@F)) $(CPPFLAGS-$(basename $(@F)))
> > +
> > +ifeq (yes,$(build-tunables))
> > +CPPFLAGS += -DTOP_NAMESPACE=glibc
> > +endif
> > +
> > override CFLAGS = -std=gnu11 -fgnu89-inline $(config-extra-cflags) \
> > $(filter-out %frame-pointer,$(+cflags)) $(+gccwarn-c) \
> > $(sysdep-CFLAGS) $(CFLAGS-$(suffix $@)) $(CFLAGS-$(<F)) \
> > @@ -1098,6 +1103,10 @@ all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
> > crypt localedata timezone rt conform debug mathvec \
> > $(add-on-subdirs) dlfcn elf
> >
> > +ifeq (yes,$(build-tunables))
> > +all-subdirs += tunables
> > +endif
> > +
> > ifndef avoid-generated
> > # sysd-sorted itself will contain rules making the sysd-sorted target
> > # depend on Depend files. But if you just added a Depend file to an
> > diff --git a/config.h.in b/config.h.in
> > index ec9c8bc..9b07580 100644
> > --- a/config.h.in
> > +++ b/config.h.in
> > @@ -231,4 +231,7 @@
> > /* PowerPC32 uses fctidz for floating point to long long conversions. */
> > #define HAVE_PPC_FCTIDZ 0
> >
> > +/* Build glibc with tunables support. */
> > +#define BUILD_TUNABLES 0
> > +
> > #endif
> > diff --git a/config.make.in b/config.make.in
> > index 05ed6ec..b158f51 100644
> > --- a/config.make.in
> > +++ b/config.make.in
> > @@ -90,6 +90,7 @@ use-nscd = @use_nscd@
> > build-hardcoded-path-in-tests= @hardcoded_path_in_tests@
> > build-pt-chown = @build_pt_chown@
> > enable-lock-elision = @enable_lock_elision@
> > +build-tunables = @build_tunables@
> >
> > # Build tools.
> > CC = @CC@
> > diff --git a/configure b/configure
> > index aa05d49..6fef02c 100755
> > --- a/configure
> > +++ b/configure
> > @@ -659,6 +659,7 @@ multi_arch
> > base_machine
> > add_on_subdirs
> > add_ons
> > +build_tunables
> > build_pt_chown
> > build_nscd
> > link_obsolete_rpc
> > @@ -773,6 +774,7 @@ enable_systemtap
> > enable_build_nscd
> > enable_nscd
> > enable_pt_chown
> > +enable_tunables
> > enable_mathvec
> > with_cpu
> > '
> > @@ -1440,6 +1442,7 @@ Optional Features:
> > --disable-build-nscd disable building and installing the nscd daemon
> > --disable-nscd library functions will not contact the nscd daemon
> > --enable-pt_chown Enable building and installing pt_chown
> > + --enable-tunables Enable tunables support
> > --enable-mathvec Enable building and installing mathvec [default
> > depends on architecture]
> >
> > @@ -3642,6 +3645,19 @@ if test "$build_pt_chown" = yes; then
> >
> > fi
> >
> > +# Check whether --enable-tunables was given.
> > +if test "${enable_tunables+set}" = set; then :
> > + enableval=$enable_tunables; build_tunables=$enableval
> > +else
> > + build_tunables=no
> > +fi
> > +
> > +
> > +if test "$build_tunables" = yes; then
> > + $as_echo "#define BUILD_TUNABLES 1" >>confdefs.h
> > +
> > +fi
> > +
> > # The abi-tags file uses a fairly simplistic model for name recognition that
> > # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
> > # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
> > diff --git a/configure.ac b/configure.ac
> > index ee7a3f1..41507ac 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -392,6 +392,16 @@ if test "$build_pt_chown" = yes; then
> > AC_DEFINE(HAVE_PT_CHOWN)
> > fi
> >
> > +AC_ARG_ENABLE([tunables],
> > + [AS_HELP_STRING([--enable-tunables],
> > + [Enable tunables support])],
> > + [build_tunables=$enableval],
> > + [build_tunables=no])
> > +AC_SUBST(build_tunables)
> > +if test "$build_tunables" = yes; then
> > + AC_DEFINE(BUILD_TUNABLES)
> > +fi
> > +
> > # The abi-tags file uses a fairly simplistic model for name recognition that
> > # can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
> > # $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
> > diff --git a/csu/init-first.c b/csu/init-first.c
> > index 77c6e1c..a4e0c54 100644
> > --- a/csu/init-first.c
> > +++ b/csu/init-first.c
> > @@ -26,6 +26,9 @@
> > #include <sys/param.h>
> > #include <sys/types.h>
> > #include <libc-internal.h>
> > +#if BUILD_TUNABLES
> > +# include <tunables/tunables.h>
> > +#endif
> >
> > #include <ldsodefs.h>
> >
> > @@ -96,6 +99,10 @@ _init (int argc, char **argv, char **envp)
> > #if defined SHARED && !defined NO_CTORS_DTORS_SECTIONS
> > __libc_global_ctors ();
> > #endif
> > +
> > +#if BUILD_TUNABLES
> > + __tunables_init (__environ);
> > +#endif
> > }
> >
> > /* This function is defined here so that if this file ever gets into
> > diff --git a/csu/libc-start.c b/csu/libc-start.c
> > index f4aa01a..7629fec 100644
> > --- a/csu/libc-start.c
> > +++ b/csu/libc-start.c
> > @@ -32,7 +32,7 @@ extern int __libc_multiple_libcs;
> > #include <tls.h>
> > #ifndef SHARED
> > # include <dl-osinfo.h>
> > -extern void __pthread_initialize_minimal (void);
> > +extern void __pthread_initialize_minimal (int, char **, char **);
> > # ifndef THREAD_SET_STACK_GUARD
> > /* Only exported for architectures that don't store the stack guard canary
> > in thread local area. */
> > @@ -193,7 +193,7 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL),
> > /* Initialize the thread library at least a bit since the libgcc
> > functions are using thread functions if these are available and
> > we need to setup errno. */
> > - __pthread_initialize_minimal ();
> > + __pthread_initialize_minimal (argc, argv, __environ);
> >
> > /* Set up the stack checker's canary. */
> > uintptr_t stack_chk_guard = _dl_setup_stack_chk_guard (_dl_random);
> > diff --git a/malloc/arena.c b/malloc/arena.c
> > index 1edb4d4..f773b10 100644
> > --- a/malloc/arena.c
> > +++ b/malloc/arena.c
> > @@ -19,6 +19,11 @@
> >
> > #include <stdbool.h>
> >
> > +#if BUILD_TUNABLES
> > +# define TUNABLE_NAMESPACE malloc
> > +# include <tunables/tunables.h>
> > +#endif
> > +
> > /* Compile-time constants. */
> >
> > #define HEAP_MIN_SIZE (32 * 1024)
> > @@ -332,6 +337,61 @@ ptmalloc_unlock_all2 (void)
> > # endif
> > #endif /* !NO_THREADS */
> >
> > +#if BUILD_TUNABLES
> > +static void
> > +tunable_set_mallopt_top_pad (const char *val)
> > +{
> > + __libc_mallopt (M_TOP_PAD, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_perturb (const char *val)
> > +{
> > + __libc_mallopt (M_PERTURB, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_mmap_max (const char *val)
> > +{
> > + __libc_mallopt (M_MMAP_MAX, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_arena_max (const char *val)
> > +{
> > + __libc_mallopt (M_ARENA_MAX, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_arena_test (const char *val)
> > +{
> > + __libc_mallopt (M_ARENA_TEST, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_trim_threshold (const char *val)
> > +{
> > + __libc_mallopt (M_TRIM_THRESHOLD, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_mmap_threshold (const char *val)
> > +{
> > + __libc_mallopt (M_MMAP_THRESHOLD, atoi (val));
> > +}
> > +
> > +static void
> > +tunable_set_mallopt_check (const char *val)
> > +{
> > + if (val[0])
> > + {
> > + __libc_mallopt (M_CHECK_ACTION, (int) (val[0] - '0'));
> > + if (check_action != 0)
> > + __malloc_check_init ();
> > + }
> > +}
> > +
> > +#else
> > /* Initialization routine. */
> > #include <string.h>
> > extern char **_environ;
> > @@ -366,6 +426,7 @@ next_env_entry (char ***position)
> >
> > return result;
> > }
> > +#endif
> >
> >
> > #ifdef SHARED
> > @@ -401,6 +462,29 @@ ptmalloc_init (void)
> >
> > thread_arena = &main_arena;
> > thread_atfork (ptmalloc_lock_all, ptmalloc_unlock_all, ptmalloc_unlock_all2);
> > +
> > +#if BUILD_TUNABLES
> > + COMPAT_TUNABLES_NAMESPACE_BEGIN (8);
> > +
> > + COMPAT_TUNABLE_REGISTER_SECURE (check, "MALLOC_CHECK_",
> > + tunable_set_mallopt_check);
> > + COMPAT_TUNABLE_REGISTER (top_pad, "MALLOC_TOP_PAD_",
> > + tunable_set_mallopt_top_pad);
> > + COMPAT_TUNABLE_REGISTER (perturb, "MALLOC_PERTURB_",
> > + tunable_set_mallopt_perturb);
> > + COMPAT_TUNABLE_REGISTER (mmap_threshold, "MALLOC_MMAP_THRESHOLD_",
> > + tunable_set_mallopt_mmap_threshold);
> > + COMPAT_TUNABLE_REGISTER (trim_threshold, "MALLOC_TRIM_THRESHOLD_",
> > + tunable_set_mallopt_trim_threshold);
> > + COMPAT_TUNABLE_REGISTER (mmap_max, "MALLOC_MMAP_MAX_",
> > + tunable_set_mallopt_mmap_max);
> > + COMPAT_TUNABLE_REGISTER (arena_max, "MALLOC_ARENA_MAX",
> > + tunable_set_mallopt_arena_max);
> > + COMPAT_TUNABLE_REGISTER (arena_test, "MALLOC_ARENA_TEST",
> > + tunable_set_mallopt_arena_test);
> > +
> > + COMPAT_TUNABLES_NAMESPACE_INIT ();
> > +#else
> > const char *s = NULL;
> > if (__glibc_likely (_environ != NULL))
> > {
> > @@ -469,6 +553,8 @@ ptmalloc_init (void)
> > if (check_action != 0)
> > __malloc_check_init ();
> > }
> > +#endif
> > +
> > void (*hook) (void) = atomic_forced_read (__malloc_initialize_hook);
> > if (hook != NULL)
> > (*hook)();
> > diff --git a/manual/install.texi b/manual/install.texi
> > index de9d270..95392ea 100644
> > --- a/manual/install.texi
> > +++ b/manual/install.texi
> > @@ -195,6 +195,11 @@ configure with @option{--disable-werror}.
> > By default for x86_64, @theglibc{} is built with vector math library.
> > Use this option to disable vector math library.
> >
> > +@item --enable-tunables
> > +Tunables support allows additional library parameters to be customized at
> > +runtime for each application. This is an experimental feature and affects
> > +startup time and is thus disabled by default.
> > +
> > @item --build=@var{build-system}
> > @itemx --host=@var{host-system}
> > These options are for cross-compiling. If you specify both options and
> > diff --git a/nptl/nptl-init.c b/nptl/nptl-init.c
> > index bdbdfed..6667801 100644
> > --- a/nptl/nptl-init.c
> > +++ b/nptl/nptl-init.c
> > @@ -38,6 +38,9 @@
> > #include <kernel-features.h>
> > #include <libc-internal.h>
> > #include <pthread-pids.h>
> > +#if BUILD_TUNABLES
> > +# include <tunables/tunables.h>
> > +#endif
> >
> > #ifndef TLS_MULTIPLE_THREADS_IN_TCB
> > /* Pointer to the corresponding variable in libc. */
> > @@ -297,7 +300,7 @@ extern void **__libc_dl_error_tsd (void) __attribute__ ((const));
> > static bool __nptl_initial_report_events __attribute_used__;
> >
> > void
> > -__pthread_initialize_minimal_internal (void)
> > +__pthread_initialize_minimal_internal (int argc, char **argv, char **envp)
> > {
> > #ifndef SHARED
> > /* Unlike in the dynamically linked case the dynamic linker has not
> > @@ -311,6 +314,10 @@ __pthread_initialize_minimal_internal (void)
> > __asm __volatile ("");
> > #endif
> >
> > +#if BUILD_TUNABLES
> > + __tunables_init (envp);
> > +#endif
> > +
> > /* Minimal initialization of the thread descriptor. */
> > struct pthread *pd = THREAD_SELF;
> > __pthread_initialize_pids (pd);
> > diff --git a/scripts/gen-tunables.awk b/scripts/gen-tunables.awk
> > new file mode 100644
> > index 0000000..ccbf789
> > --- /dev/null
> > +++ b/scripts/gen-tunables.awk
> > @@ -0,0 +1,84 @@
> > +# Generate tunable-list.h from tunables.list
> > +
> > +BEGIN {
> > + ns=""
> > + top_ns=""
> > +}
> > +
> > +$2 == "{" {
> > + if (top_ns == "") {
> > + top_ns = $1
> > + }
> > + else if (ns == "") {
> > + ns = $1
> > + count = 0
> > + }
> > + else {
> > + printf ("Unexpected occurrence of '{' inside a namespace: %s:%d\n",
> > + FILENAME, FNR)
> > + exit 1
> > + }
> > +
> > + next
> > +}
> > +
> > +$1 == "}" {
> > + if (ns != "") {
> > + ns = ""
> > + }
> > + else if (top_ns != "") {
> > + top_ns = ""
> > + }
> > + else {
> > + printf ("syntax error: extra }: %s:%d\n", FILENAME, FNR)
> > + exit 1
> > + }
> > + next
> > +}
> > +
> > +{
> > + if (ns == "") {
> > + print "Invalid tunable outside a namespace"
> > + exit 1
> > + }
> > + val[top_ns][ns][count] = $1
> > + count = count + 1
> > +}
> > +
> > +END {
> > + if (ns != "") {
> > + print "Unterminated namespace. Is a closing brace missing?"
> > + exit 1
> > + }
> > +
> > + print "/* Print a full tunable enum name. */"
> > + print "#include <stddef.h>"
> > + print "#define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)"
> > + print "#define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id\n"
> > +
> > + print "/* Full name for a tunable is top_ns.tunable_ns.id. */"
> > + print "#define TUNABLE_NAME_S(top,ns,id) #top \".\" #ns \".\" #id\n"
> > +
> > + print "typedef enum"
> > + print "{"
> > + for (t in val) {
> > + for (n in val[t]) {
> > + for (c in val[t][n]) {
> > + printf (" TUNABLE_ENUM_NAME(%s, %s, %s),\n", t, n, val[t][n][c]);
> > + }
> > + }
> > + }
> > + print "} tunable_id_t;\n"
> > + print "#ifdef TUNABLES_INTERNAL"
> > + print "static tunable_t tunable_list[] = {"
> > + for (t in val) {
> > + for (n in val[t]) {
> > + for (c in val[t][n]) {
> > + printf (" {TUNABLE_NAME_S(%s, %s, %s), NULL, NULL, false},\n",
> > + t, n, val[t][n][c], t, n, val[t][n][c]);
> > + }
> > + }
> > + }
> > + print "};"
> > + print "#endif"
> > +}
> > diff --git a/tunables/Makefile b/tunables/Makefile
> > new file mode 100644
> > index 0000000..44439f6
> > --- /dev/null
> > +++ b/tunables/Makefile
> > @@ -0,0 +1,33 @@
> > +# Copyright (C) 2016 Free Software Foundation, Inc.
> > +# This file is part of the GNU C Library.
> > +
> > +# The GNU C Library is free software; you can redistribute it and/or
> > +# modify it under the terms of the GNU Lesser General Public
> > +# License as published by the Free Software Foundation; either
> > +# version 2.1 of the License, or (at your option) any later version.
> > +
> > +# The GNU C Library is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > +# Lesser General Public License for more details.
> > +
> > +# You should have received a copy of the GNU Lesser General Public
> > +# License along with the GNU C Library; if not, see
> > +# <http://www.gnu.org/licenses/>.
> > +
> > +#
> > +# Makefile for tunables.
> > +#
> > +subdir := tunables
> > +
> > +include ../Makeconfig
> > +
> > +routines = tunables
> > +
> > +$(objpfx)tunables.os: tunable-list.h
> > +
> > +tunable-list.h: $(..)scripts/gen-tunables.awk tunables.list
> > + $(AWK) -f $^ > $@.tmp
> > + mv $@{.tmp,}
> > +
> > +include ../Rules
> > diff --git a/tunables/README b/tunables/README
> > new file mode 100644
> > index 0000000..02a3f7a
> > --- /dev/null
> > +++ b/tunables/README
> > @@ -0,0 +1,93 @@
> > + TUNABLE FRAMEWORK
> > + =================
> > +
> > +The tunable framework allows modules within glibc to register variables that
> > +may be tweaked through an environment variable or an API call. It aims to
> > +enforce a strict namespace rule to bring consistency to naming of these tunable
> > +environment variables across the project.
> > +
> > +ADDING A NEW TUNABLE
> > +--------------------
> > +
> > +The TOP_NAMESPACE is defined by default as 'glibc' and it may be overridden in
> > +distributions for specific tunables if they want to add their own tunables.
> > +Downstream implementations are discouraged from using the 'glibc' namespace for
> > +tunables they don't already have consensus to push upstream.
> > +
> > +There are two steps to adding a tunable:
> > +
> > +1. Add a tunable ID:
> > +
> > +Modules that wish to use the tunables interface must define the
> > +TUNABLE_NAMESPACE macro. Following this, for each tunable you want to
> > +register, call the TUNABLE_REGISTER macro.
> > +
> > +The TUNABLE_REGISTER macro takes the following arguments:
> > +
> > +- id: The short name of the tunable. It will be concatenated with
> > + the TOP_NAMESPACE and TUNABLE_NAMESPACE to build the full ID of
> > + the tunable.
> > +
> > +- setter: A function that accepts a string input and initializes the
> > + tunable variable. The prototype of the function should be as
> > + follows:
> > +
> > + void setter_func (const char *)
> > +
> > +2. Add to tunables.list:
> > +
> > +This file nests tunable ids inside their tunable namespace within curly braces
> > +and multiple such namespaces are in turn nested inside their top namespaces in
> > +a similar manner. The malloc and resolv snippets for example could look like
> > +this:
> > +
> > +glibc {
> > + malloc {
> > + check
> > + top_pad
> > + }
> > + resolv {
> > + secure_dns
> > + }
> > +}
> > +
> > +Add your tunable in this hierarchy using the proper nesting for your namespace
> > +and top namespace. If the top namespace is not glibc, create a new snippet
> > +with its own hierarchy, separate from the glibc hierarchy.
> > +
> > +ADDING A LEGACY TUNABLE
> > +-----------------------
> > +
> > +One may add tunables for variables that are currently controlled by users with
> > +environment variables. Multiple such variables should be defined together for
> > +better performance by using the COMPAT_TUNABLES_NAMESPACE_BEGIN macro, which
> > +accepts a numeric parameter with the number of tunables you intend to register
> > +in the block.
> > +
> > +Individual tunables are then registered using the COMPAT_TUNABLE_REGISTER and
> > +COMPAT_TUNABLE_REGISTER_SECURE macros. They take the following parameters:
> > +
> > +- id: The tunable ID, similar to the normal way of adding a tunable
> > +
> > +- env: The name of the legacy environment variable so that one may
> > + read its value and use it.
> > +
> > +- setter: The setter function, similar to the normal way of adding a
> > + tunable.
> > +
> > +COMPAT_TUNABLE_REGISTER reads the environment variable only for non-setuid
> > +binaries, whereas COMPAT_TUNABLE_REGISTER_SECURE always attempts to read the
> > +environment variable. Use COMPAT_TUNABLE_REGISTER_SECURE with care.
> > +
> > +You will also have to add the tunable id to tunables/tunables.list.
> > +
> > +DO NOT ADD NEW TUNABLES USING THIS MECHANISM!
> > +
> > +FUTURE WORK
> > +-----------
> > +
> > +The framework currently only allows a one-time initialization of variables
> > +through environment variables and in some cases, modification of variables via
> > +an API call. A future goal for this project is to allow tweaking of some
> > +values in a running process, possibly through some kind of shared memory
> > +mechanism.
> > diff --git a/tunables/Versions b/tunables/Versions
> > new file mode 100644
> > index 0000000..4c68b0e
> > --- /dev/null
> > +++ b/tunables/Versions
> > @@ -0,0 +1,8 @@
> > +# Exports from tunables should only be in the GLIBC_PRIVATE namespace.
> > +libc {
> > + GLIBC_PRIVATE {
> > + __compat_tunables_init_envvars;
> > + __tunables_init;
> > + __tunable_register;
> > + }
> > +}
> > diff --git a/tunables/tunable-list.h b/tunables/tunable-list.h
> > new file mode 100644
> > index 0000000..2b1caa4
> > --- /dev/null
> > +++ b/tunables/tunable-list.h
> > @@ -0,0 +1,32 @@
> > +/* Print a full tunable enum name. */
> > +#include <stddef.h>
> > +#define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)
> > +#define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id
> > +
> > +/* Full name for a tunable is top_ns.tunable_ns.id. */
> > +#define TUNABLE_NAME_S(top,ns,id) #top "." #ns "." #id
> > +
> > +typedef enum
> > +{
> > + TUNABLE_ENUM_NAME(glibc, malloc, check),
> > + TUNABLE_ENUM_NAME(glibc, malloc, top_pad),
> > + TUNABLE_ENUM_NAME(glibc, malloc, perturb),
> > + TUNABLE_ENUM_NAME(glibc, malloc, mmap_threshold),
> > + TUNABLE_ENUM_NAME(glibc, malloc, trim_threshold),
> > + TUNABLE_ENUM_NAME(glibc, malloc, mmap_max),
> > + TUNABLE_ENUM_NAME(glibc, malloc, arena_max),
> > + TUNABLE_ENUM_NAME(glibc, malloc, arena_test),
> > +} tunable_id_t;
> > +
> > +#ifdef TUNABLES_INTERNAL
> > +static tunable_t tunable_list[] = {
> > + {TUNABLE_NAME_S(glibc, malloc, check), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, top_pad), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, perturb), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, mmap_threshold), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, trim_threshold), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, mmap_max), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, arena_max), NULL, NULL, false},
> > + {TUNABLE_NAME_S(glibc, malloc, arena_test), NULL, NULL, false},
> > +};
> > +#endif
> > diff --git a/tunables/tunables.c b/tunables/tunables.c
> > new file mode 100644
> > index 0000000..8834aff
> > --- /dev/null
> > +++ b/tunables/tunables.c
> > @@ -0,0 +1,163 @@
> > +/* The tunable framework. See the README to know how to use the tunable in
> > + a glibc module.
> > +
> > + Copyright (C) 2016 Free Software Foundation, Inc.
> > + This file is part of the GNU C Library.
> > +
> > + The GNU C Library is free software; you can redistribute it and/or
> > + modify it under the terms of the GNU Lesser General Public
> > + License as published by the Free Software Foundation; either
> > + version 2.1 of the License, or (at your option) any later version.
> > +
> > + The GNU C Library is distributed in the hope that it will be useful,
> > + but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + Lesser General Public License for more details.
> > +
> > + You should have received a copy of the GNU Lesser General Public
> > + License along with the GNU C Library; if not, see
> > + <http://www.gnu.org/licenses/>. */
> > +
> > +#include <stddef.h>
> > +#include <stdlib.h>
> > +#include <string.h>
> > +#include <stdbool.h>
> > +#include <unistd.h>
> > +#include <sys/param.h>
> > +
> > +extern char **__environ;
> > +
> > +#define TUNABLES_INTERNAL 1
> > +#include "tunables.h"
> > +
> > +/* We avoid calling into the C library as much as we can, especially functions
> > + that we know could use tunables in future for some reason or the other. For
> > + example, we call mmap directly instead of malloc because malloc uses
> > + tunables. Similarly, the string functions may use tunables in future to
> > + select optimal routines and we want to make sure we stay out of the way and
> > + not invoke their initializers before the tunable list has been
> > + initialized. */
> > +
> > +static void
> > +t_memcpy (char *dest, const char *src, size_t len)
> > +{
> > + for (size_t i = 0; i < len; i++)
> > + dest[i] = src[i];
> > +}
> > +
> > +static int
> > +t_strcmp (const char *a, const char *b)
> > +{
> > + size_t i = 0;
> > +
> > + for (i = 0; *a != '\0' && *b != '\0'; i++, a++, b++)
> > + if (*a != *b)
> > + return *a - *b;
> > +
> > + return *a - *b;
> > +}
> > +
> > +static bool
> > +get_next_env (char ***envp, char **name, size_t *namelen, char **val)
> > +{
> > + char **ev = *envp;
> > +
> > + while (ev != NULL && *ev != '\0')
> > + {
> > + char *envline = *ev;
> > + int len = 0;
> > +
> > + while (envline[len] != '\0' && envline[len] != '=')
> > + len++;
> > +
> > + /* Just the name and no value, go to the next one. */
> > + if (envline[len] == '\0')
> > + continue;
> > +
> > + *name = envline;
> > + *namelen = len;
> > + *val = &envline[len + 1];
> > + *envp = ++ev;
> > +
> > + return true;
> > + }
> > +
> > + return false;
> > +}
> > +
> > +/* Initialize tunables from the GLIBC_TUNABLES environment variable. The
> > + variable is set as colon separated name=value pairs. This routine may be
> > + called from the constructor of either libpthread.so or lic.so. Constructor
> > + calls for libraries are protected by the dl_load_lock, so this routine won't
> > + be called concurrently from diferent threads. */
> > +void
> > +__tunables_init (char **envp)
> > +{
> > + /* Empty for now. */
> > +}
> > +libc_hidden_def (__tunables_init)
> > +
> > +/* Initialize all tunables using its legacy environment variable values whose
> > + names are passed in ENVVARS. */
> > +void
> > +__compat_tunables_init_envvars (struct compat_tunable_env *envvars, int count)
> > +{
> > + /* Traverse through the environment to find environment variables we may need
> > + to set. */
> > + char **envp = __environ;
> > + char *envname = NULL;
> > + char *envval = NULL;
> > + size_t len = 0;
> > +
> > + while (get_next_env (&envp, &envname, &len, &envval))
> > + {
> > + int init_count = 0;
> > + for (int i = 0; i < count; i++)
> > + {
> > + tunable_id_t t = envvars[i].id;
> > + tunable_t *cur = &tunable_list[t];
> > +
> > + /* Skip over tunables that have already been initialized. */
> > + if (cur->initialized)
> > + {
> > + init_count++;
> > + continue;
> > + }
> > +
> > + const char *name = envvars[i].env;
> > + char *ename = alloca (len + 1);
> > +
> > + t_memcpy (ename, envname, len);
> > + ename[len] = '\0';
> > +
> > + /* We have a match. Initialize and move on to the next line. */
> > + if (t_strcmp (ename, name) == 0)
> > + {
> > + cur->val = envval;
> > + cur->set (cur->val);
> > + cur->initialized = true;
> > + break;
> > + }
> > + }
> > +
> > + /* All of the tunable envvars have been initialized. */
> > + if (count == init_count)
> > + break;
> > + }
> > +}
> > +libc_hidden_def (__compat_tunables_init_envvars)
> > +
> > +/* Initialize a tunable and set its value. */
> > +void
> > +__tunable_register (tunable_id_t id, tunable_setter_t set_func)
> > +{
> > + tunable_t *cur = &tunable_list[id];
> > +
> > + cur->set = set_func;
> > + if (cur->val != NULL)
> > + {
> > + set_func (cur->val);
> > + cur->initialized = true;
> > + }
> > +}
> > +libc_hidden_def (__tunable_register)
> > diff --git a/tunables/tunables.h b/tunables/tunables.h
> > new file mode 100644
> > index 0000000..6aef30e
> > --- /dev/null
> > +++ b/tunables/tunables.h
> > @@ -0,0 +1,110 @@
> > +/* The tunable framework. See the README to know how to use the tunable in
> > + a glibc module.
> > +
> > + Copyright (C) 2016 Free Software Foundation, Inc.
> > + This file is part of the GNU C Library.
> > +
> > + The GNU C Library is free software; you can redistribute it and/or
> > + modify it under the terms of the GNU Lesser General Public
> > + License as published by the Free Software Foundation; either
> > + version 2.1 of the License, or (at your option) any later version.
> > +
> > + The GNU C Library is distributed in the hope that it will be useful,
> > + but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + Lesser General Public License for more details.
> > +
> > + You should have received a copy of the GNU Lesser General Public
> > + License along with the GNU C Library; if not, see
> > + <http://www.gnu.org/licenses/>. */
> > +
> > +#include <stdbool.h>
> > +
> > +typedef void (*tunable_setter_t) (const char *);
> > +
> > +/* A tunable. */
> > +struct _tunable
> > +{
> > + const char *name;
> > + const char *val;
> > + tunable_setter_t set;
> > + bool initialized;
> > +};
> > +
> > +typedef struct _tunable tunable_t;
> > +
> > +#include "tunable-list.h"
> > +
> > +struct compat_tunable_env
> > +{
> > + tunable_id_t id;
> > + const char *env;
> > +};
> > +
> > +
> > +extern void __compat_tunables_init_envvars (struct compat_tunable_env *, int);
> > +extern void __tunable_register (tunable_id_t, tunable_setter_t);
> > +extern void __tunables_init (char **);
> > +
> > +/* Avoid a PLT reference when called from within libc.so. */
> > +libc_hidden_proto (__compat_tunables_init_envvars)
> > +libc_hidden_proto (__tunable_register)
> > +libc_hidden_proto (__tunables_init)
> > +
> > +/* Register a tunable ID in namespace NS inside the glibc top namespace with
> > + SET as the setter function. */
> > +#define TUNABLE_REGISTER(ns,id,set) \
> > + TUNABLE_REGISTER_FULL (TOP_NAMESPACE, ns, id, set)
> > +
> > +/* Register a tunable ID in namespace NS inside the TNS top namespace with
> > + SET as the setter function. */
> > +#define TUNABLE_REGISTER_FULL(tns,ns,id,set) \
> > + __tunable_register (TUNABLE_ENUM_NAME (tns, ns, id), (set))
> > +
> > +
> > +
> > +/* COMPATIBILITY INTERFACE. These macros should be used to define tunables for
> > + variables that are currently being controlled using environment variables.
> > + New tunables must not use this interface. */
> > +
> > +#define _ADD_COMPAT_TUNABLE_ENV(__id,__env) \
> > +({ \
> > + envvars[envvars_cnt].id = TUNABLE_ENUM_NAME (TOP_NAMESPACE, \
> > + TUNABLE_NAMESPACE, __id); \
> > + envvars[envvars_cnt++].env = (__env); \
> > +})
> > +
> > +/* Start registering tunables in the current namespace. */
> > +#define COMPAT_TUNABLES_NAMESPACE_BEGIN(size) \
> > + { \
> > + struct compat_tunable_env envvars[size]; \
> > + int envvars_cnt = 0;
> > +
> > +/* Register a tunable. This macro validates that the call is OK and then calls
> > + tunable_init to do the real work of adding the tunable and setting its value
> > + based on its environment variable(s). */
> > +#define COMPAT_TUNABLE_REGISTER(id,env,set) \
> > +({ \
> > + assert (envvars_cnt < (sizeof (envvars) \
> > + / sizeof (struct compat_tunable_env))); \
> > + if (!__libc_enable_secure) \
> > + _ADD_COMPAT_TUNABLE_ENV (id, env); \
> > + TUNABLE_REGISTER (TUNABLE_NAMESPACE, id, set); \
> > +})
> > +
> > +/* Does exactly the same thing as TUNABLE_REGISTER, except that it allows the
> > + tunable to look for environment variable values even for setuid binaries.
> > + This is a separate macro and not just another parameter in TUNABLE_REGISTER
> > + to avoid accidentally setting a secure flag where it is not required. */
> > +#define COMPAT_TUNABLE_REGISTER_SECURE(id,env,set) \
> > +({ \
> > + assert (envvars_cnt < (sizeof (envvars) \
> > + / sizeof (struct compat_tunable_env))); \
> > + _ADD_COMPAT_TUNABLE_ENV(id, env); \
> > + TUNABLE_REGISTER (TUNABLE_NAMESPACE, id, set); \
> > +})
> > +
> > +/* Initialize tunables in the namespace. */
> > +#define COMPAT_TUNABLES_NAMESPACE_INIT() \
> > + __compat_tunables_init_envvars (envvars, envvars_cnt); \
> > + }
> > diff --git a/tunables/tunables.list b/tunables/tunables.list
> > new file mode 100644
> > index 0000000..f1335cc
> > --- /dev/null
> > +++ b/tunables/tunables.list
> > @@ -0,0 +1,12 @@
> > +glibc {
> > + malloc {
> > + check
> > + top_pad
> > + perturb
> > + mmap_threshold
> > + trim_threshold
> > + mmap_max
> > + arena_max
> > + arena_test
> > + }
> > +}
> > --
> > 2.5.0
> >