glibc aio performance

[Amos P Waterland]

Hello, I have been looking at the glibc asynchronous I/O implementation,
and have run into a bit of an issue.

When I run IOzone (an open source filesystem benchmark tool) with AIO
enabled, it reports write KB/s throughput on the order of 45 times slower
than that reported without AIO.

% iozone -i0 -i1 #run just the write and read tests
[snip]
KB  reclen   write rewrite    read    reread [snip]
512       4  101769  178334   331595   345702
% iozone -i0 -i1 -k128  #do same, but use no-bcopy aio
[snip]
KB  reclen   write rewrite    read    reread [snip]
512       4    2232   47210   121874   103372

I have looked at the source code for the glibc implementation, and it is
not obvious to me why keeping a thread pool, each of whose contituents
perform a pread(2) or pwrite(2), should be so much slower than synchronous
I/O.  I looked at the source code for IOzone, and found that it uses
libasync.c for AIO, but could find no obvious performance problems in its
code.

So my question is: Might there be a problem with the way IOzone is using
glibc's implementation of AIO, or is glibc's implementation known to have
performance problems?

Amos Waterland