This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Robust getpid caching via MADV_WIPEONFORK?
- From: Josh Triplett <josh at joshtriplett dot org>
- To: libc-alpha at sourceware dot org
- Date: Wed, 22 Nov 2017 12:34:04 -0800
- Subject: Robust getpid caching via MADV_WIPEONFORK?
- Authentication-results: sourceware.org; auth=none
[Bcced to other potentially interested folks.]
As discussed in various places, glibc removed the getpid() cache in
2.25, since it was not robust against all possible ways to fork a
process.
Linux 4.14 added MADV_WIPEONFORK, which robustly ensures that a page
gets wiped to zero on any possible process fork. The commit message
adding it even mentions, as a use case:
> - systemd/pulseaudio API checks (fail after fork) (replacing a getpid
> check, which is too slow without a PID cache)
Given that, I wanted to start a thread about the idea of making
getpid() caching, and for that matter other potential uses of
pthread_atfork(), robust using MADV_WIPEONFORK when available.
I don't necessarily want to advocate this; rather, since it seems likely
that other applications may wish to do things like this, I wanted to
collect some information and discuss whether it makes sense or not.
I wrote a simple test program, attached (warning: quick hack), that
benchmarks the stock getpid() versus a trivial cached version of
getpid() using MADV_WIPEONFORK (marked as "noinline" to simulate
providing it as a library function), just to get a rough idea. This
produced the following numbers on my system:
mmap: 3292.000000 ns
madvise: 3273.000000 ns
uncached getpid: 100000000 calls in 4569540458.000 ns; 45.695 ns/call
cached getpid: 100000000 calls in 132862952.000 ns; 1.329 ns/call
That's a significant speedup per call, but that savings only pays off if
the program calls getpid() more than ~150 times, or ~75 times if the
separate mmap can be avoided.
Given that, I don't think it makes sense for glibc to take this approach
in the standard getpid(). For any program that doesn't call getpid()
extensively, this seems like a pessimization, and many such programs can
do such caching themselves without worrying about an unexpected fork().
I think only specialized library code would ever want to do this.
Hopefully these numbers will help anyone looking to implement such
caching in their own code.
- Josh Triplett
#include <err.h>
#include <stddef.h>
#include <stdio.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <time.h>
#include <unistd.h>
#ifndef MADV_WIPEONFORK
#define MADV_WIPEONFORK 18
#endif
static pid_t *cached_getpid_page;
__attribute__((noinline))
static pid_t cached_getpid(void)
{
pid_t pid = *cached_getpid_page;
if (!pid)
*cached_getpid_page = pid = getpid();
return pid;
}
static struct timespec get_time(void)
{
struct timespec t;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &t) < 0)
err(1, "clock_gettime");
return t;
}
static double to_nsec(struct timespec t)
{
return t.tv_sec * 1000000000 + t.tv_nsec;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &t) < 0)
err(1, "clock_gettime");
printf("%llu.%09lu\n", (unsigned long long)t.tv_sec, t.tv_nsec);
}
static const int ITERATIONS = 100000000;
int main(void)
{
struct timespec start, end;
double duration;
start = get_time();
void *page = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (page == MAP_FAILED)
err(1, "mmap");
end = get_time();
duration = to_nsec(end) - to_nsec(start);
printf("mmap: %f ns\n", duration);
start = get_time();
if (madvise(page, 4096, MADV_WIPEONFORK) < 0)
err(1, "madvise");
cached_getpid_page = page;
end = get_time();
duration = to_nsec(end) - to_nsec(start);
printf("madvise: %f ns\n", duration);
start = get_time();
for (int i = 0; i < ITERATIONS; i++)
getpid();
end = get_time();
duration = to_nsec(end) - to_nsec(start);
printf("uncached getpid: %d calls in %14.3f ns; %7.3f ns/call\n", ITERATIONS, duration, duration/ITERATIONS);
start = get_time();
for (int i = 0; i < ITERATIONS; i++)
cached_getpid();
end = get_time();
duration = to_nsec(end) - to_nsec(start);
printf(" cached getpid: %d calls in %14.3f ns; %7.3f ns/call\n", ITERATIONS, duration, duration/ITERATIONS);
return 0;
}