This is the mail archive of the libc-help@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Serialization of AIO requests to the same fd in posix-aio


Hi Ulrich,

In sysdeps/pthread/aio_misc.c:__aio_enqueue_request(), there is a check to see if an existing thread is working on the same file descriptor as the request being enqueued. If it is, it queues it to run after the existing request instead of possibly spawning a new thread.

The net effect is that submitting multiple requests to the same file descriptor are always serialized. This significantly affects performance.

The comment beneath this check says:

     /* The current file descriptor is worked on.  It makes no sense
    to start another thread since this new thread would fight
    with the running thread for the resources.  But we also cannot
    say that the thread processing this desriptor shall immediately
    after finishing the current job process this request if there
    are other threads in the running queue which have a higher
    priority.  */

I don't understand what "fight with the running thread" is supposed to suggest. Is this suggesting that there is some fundamental race condition? Is it assuming that two IO requests cannot happen at the same time at the kernel level? Since the threads end up using pread/pwrite, it should be safe to have two request executing simultaneously.

Currently, we use posix-aio for disk IO in QEMU/KVM. To achieve good performance, we have to implement a fd pool using dup() to work around this check. The difference between using the pool and not using the pool is around 400% in write performance.

Can you further elaborate on what this check is trying to prevent?

Thanks,

Anthony Liguori


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]