Tuning Library Runtime Behavior

WORK IN PROGRESS

The following material is a work in progress and should not be considered complete or ready for public use.

1. Why?

No set of library defaults is appropriate for all workloads.

The GNU C Library makes assumptions on behalf of the user and provides a specific runtime behaviour that may not match the user workload or requirements.

For example the NPTL implementation sets a fixed cache size of 40MB for the re-use of thread stacks. Is it possible that this is correct under all workloads? Average workloads? This default was set 10 years ago and has not been revisited.

I propose we expose some of the library internals as tunable runtime parameters that our users and developers can use to tune the library. Developers would use them to achieve optimal mean performance for all users, while a single advanced user might use it to get the best performance from their application.

To reiterate:

We have immediate short-term needs today to expose library internals as tunable parameters, in particular:

2. How?

3. Desgin examples

3.1. Example: Some properties read at startup others continually via a global pointer

The only feasible design today is to create a global pointer that points to a structure that contains all tunnables for the entire library. At startup certain values of this structure are used for IFUNC selection and to initialize library-wide values that need early initialization. Later some values which can be dynamically changed may also be read via this global pointer e.g. default thread stack size. We document each property and if it's applied at startup, or if it is read at ever use. Startup properties could only be set via env vars or an admin sysconfig file read at startup.

3.2. Example: Fully dynamic properties via a global pointer

This is only a toy example of how one might use a global pointer, and a lockless algorithm, to push and pop tunable contexts for the entire library to use. The entire library would need to reference tunables via some levels of indirection through the global pointer (previously just referenced the global pointer).

For example:

/* A definition of a tunable is a name/value tuple (for now).  */
struct __tunable {
  char *tunable;
  char *value;
};
typedef struct __tunable tunable;

/* The tunables have IDs that we use to index into the tunable table
   for each context.  */
enum {
  GNU_LIBC_PTHREAD_DEFAULT_STACKSIZE = 0,
  GNU_LIBC_PTHREAD_STACK_CACHESIZE = 1,
  ...
  GNU_LIBC_MAX_TUNABLE = 100
};

/* A context contains a set of tunables.  */
struct __tunable_context {
  char *id;
  tunable tlist[GNU_LIBC_MAX_TUNABLE];
  tunable_context *previous; 
};
typedef struct __tunable_context tunable_context;

/* Hidden pointer to active context in the library.  */
tunable_context *__default_tunable_context attribute_hidden;

/* Create a context from the current active context and call it ID.  */
tunable_context *create_tunable_context_np (const char *id);
int destroy_tunable_context_np (tunable_context *context);

/* Set a tunable for a context.  */
int set_tunable_np (tunable_context *context, const char *tunable, const char *value);
const char *get_tunable_np (tunable_context *context, const char *tunable);

/* Push or pop a context. Overrides the previous context.  */
int push_tunable_context_np (tunable_context *context);
tunable_context *pop_tunable_context_np (void);

/* Get the list of all tunables currently available.  */
int list_tunables_np (char **tunables, int *size);

e.g.

tunable_context *ctx = create_tunable_context_np ();
if (set_tunable_np (ctx, "GNU_LIBC_PTHREAD_DEFAULT_STACKSIZE", "1048576") != 0)
  {
    /* Error handling.  */
  }
if (push_tunable_context_np (ctx) != 0)
  {
    /* Error handling.  */
  }
/* Do work with context active.  */
if (pop_tunable_context_np () == NULL)
  {
    /* Error handling.  */
  }
/* Restores previous context.  */

Per-process as an env var:

export GNU_LIBC_$tunable=$value

Equivalent to calling the following at startup:

tunable_context *ctx = create_tunable_context_np (NULL);
set_tunable_np (ctx, "GNU_LIBC_$tunable", "$value");
push_tunable_context_np (ctx);

Per-named-context as a env-var:

export GNU_LIBC_$tunable_$id=$value

Equivalent to calling the following at startup:

tunable_context *ctx = create_tunable_context_np ("$id");
set_tunable_np (ctx, "GNU_LIBC_$tunable", "$value");
push_tunable_context_np (ctx);

Where:

Notes:

4. Next steps

4.1. Collect all globals

As recommended in Cauldron 2013 we need to bring together a global internal private structure first that contains all of the globals one might want to modify. That way we can see what is actually tunnable.

4.2. Analyze env vars currently in use

Analyzing currenct use of glibc env vars. Currently not complete. Currently contains env vars from auxiliary libraries.