This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
BUG REPORT: aio with fork
- From: Rajat Kateja <rkateja at cmu dot edu>
- To: libc-alpha at sourceware dot org
- Date: Wed, 13 Jul 2016 10:51:57 -0700
- Subject: BUG REPORT: aio with fork
- Authentication-results: sourceware.org; auth=none
Hi,
I think I have hit a bug in the AIO system. I contacted the linux-aio
mailing list but was redirected here because I am using posix-aio.
Please see the following bug report:
One line description: Doing AIO concurrently before and after a fork
causes the IOs after the fork to be never queued.
Full Description: Consider the scenario.
1. A program issues an aio_read/aio_write
2. After the completion of the aio, the program forks
3. The child process issues an aio_read/aio_write
The aio code (from aio_misc.c) maintains a global variable
idle_thread_count. Also, aio threads idle for a period of 1 second (by
default) after completing the IO operation. During this one second,
the parent forks and the child gets the copy of the idle_thread_count
from the global data segment. When the child issues a fresh aio
request, the code sees that the idle_thread_count is greater than one
and signals a condition variable to wake up the thread. But of course,
there if no thread in the child process. Hence the AIO stays in the
EINPROGRESS state forever.
I think this might be fixed if the return value form the conditional
signalling is checked. Doing an aio_init in the child also doesn't
help because that function is basically a no-op if a global pointer
(pool) is not null. In the above scenario, since the parent had also
done some AIO, the global pointer is not null, and aio_init in the
child does not help. Moreover, the aio_init does not reset the
idle_thread_count, even if the global pool pointer is NULL. Another
possible solution is to improve the aio_init function and specify in
the man page that doing AIO with fork requires calling the aio_init
function in the child. Yet another solution is to allow the user to
set 0 as the aio_idle_timeout in the aio_init function. Then the
parent process can do the aio_init and no thread would ever be idle.
Keyword: AIO system, fork
Kernel: Linux version 4.4.0-21-generic (buildd@lgw01-21) (gcc version
5.3.1 20160413 (Ubuntu 5.3.1-14ubuntu2) ) #37-Ubuntu SMP Mon Apr 18
18:33:37 UTC 2016
Please find a minimal program to recreate the error below (Could not
send as an attachment, the mail delivery failed). It first creates and
reads from a file in the parent, then it spawns a child which does the
same thing for a different file. The child's aio never completes.
<code>
#include <stdio.h>
#include <stdlib.h>
#include <aio.h>
#include <sys/types.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
void child_process();
void main()
{
int fd = open("file", O_CREAT | O_RDWR, S_IRWXU);
ftruncate(fd, 4096);
void *buffer = malloc(4096);
struct sigevent aio_sigevent;
memset(&aio_sigevent, 0, sizeof(struct sigevent));
aio_sigevent.sigev_notify = SIGEV_NONE;
struct aiocb aio_cb;
memset(&aio_cb, 0, sizeof(struct aiocb));
aio_cb.aio_sigevent = aio_sigevent;
aio_cb.aio_fildes = fd;
aio_cb.aio_offset = 0;
aio_cb.aio_buf = buffer;
aio_cb.aio_nbytes = 4096;
aio_cb.aio_lio_opcode = LIO_READ;
aio_read(&aio_cb);
while(aio_error(&aio_cb) != 0)
{
}
pid_t child_pid = fork();
if(child_pid == 0)
{
child_process();
_exit(0);
}
else
{
wait(NULL);
//never returns from wait
}
}
void child_process()
{
int fd = open("file_child", O_CREAT | O_RDWR, S_IRWXU);
ftruncate(fd, 4096);
printf("child\n");
void *buffer = malloc(4096);
struct sigevent aio_sigevent;
aio_sigevent.sigev_notify = SIGEV_NONE;
struct aiocb aio_cb;
aio_cb.aio_sigevent = aio_sigevent;
aio_cb.aio_fildes = fd;
aio_cb.aio_offset = 0;
aio_cb.aio_buf = buffer;
aio_cb.aio_nbytes = 4096;
aio_cb.aio_lio_opcode = LIO_READ;
aio_read(&aio_cb);
while(aio_error(&aio_cb) != 0)
{
//Never breaks out of this loop
}
}
</code>
--
Rajat Kateja
Graduate Student
Parallel Data Lab
Carnegie Mellon University