Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

Hi Corinna,

On Tue, 12 Dec 2017, Corinna Vinschen wrote:
> On Dec 12 01:11, Mark Geisert wrote:
>> I've got a proof-of-concept implementation working but haven't integrated it
>> into my local Cygwin source tree yet.  This implementation uses a pool of
>> pthreads to issue reads and writes asynchronously.  Using threads allows to
>> operate transparently with the usual Cygwin-supplied file
>> descriptors/handles that are set up for synchronous-only operation.
>>
>> If I were to use Win32 overlapped i/o, as Corinna suggested as a
>> possibility, there would have to be some supporting changes to Cygwin
>> internals, I think.  Either file descriptors would have to always permit
>> overlapped usage, or open() and/or fcntl() would have to allow some new flag
>> to specify overlapped capability when creating a new fd/handle.
>
> If you use threads, you will have to change your code to use cygthreads
> instead of pthreads.  Pthreads are a userspace concept, cygthreads are
> used for internal tasks.

Understood in general.  I had originally implemented a libaio that was 
built on top of Cygwin, so there was an "app" viewpoint in my thinking. 
But I'm coming to understand what you're talking about: really doing aio_* 
and lio_* as Cygwin syscalls using the same capabilities within Cygwin 
that other syscalls use.  So cygthreads it is, only if they're needed.

> However, using async IO, the only change necessary would be to change
> prw_open to open an async handle, and pread/pwrite to wait for the
> result if the current I/O is NOT async.  It may be necessary to add a
> parameter or two to the pread/pwrite fhandler methods, but all the rest
> of the functionality could be in the callers.
>
> Please stop thinking in Win32 overlapped I/O.  I have no idea why
> Microsoft uses this term.  Under the ntdll hood, a handle is either
> synchronous (Keeping track of file position, waiting for read/write to
> complete) or asynchronous (Not keeping track of file position, returning
> STATUS_PENDING rather than waiting for completion, based on using the
> FILE_SYNCHRONOUS_IO_{NON}ALERT.
>
> On async handles, you can wait for an event object to be signalled, or
> you can ask the NtReadFile/NtWriteFile functions to call a completion
> routine.

OK.  I understand.

>> There is an O_ASYNC flag in some non-POSIX Unixes but what little
>> documentation
>
> No, if we had to create a new open flag for this functionality it would
> be wrong.
>
>> need here.  It's used to signal (via SIGIO arrival) that space is available
>> for a write, or that data is available for a read.  What it would need to do
>> for aio_* semantics is to signal that a write or a read has completed.
>
> Yes, but with an arbitrary signal.  Alternatively aio can start a
> pthread on completion, basically all the stuff sigevent is supposed to
> handle.
>
>> I'm also unsure one can add overlapped capability to an existing Win32
>> handle.
>
> That's what prw_open is for.

Ah, the light of enlightenment has lit.

>> Wouldn't it be easier (less modification of existing code) to just use the
>> thread model?
>
> Not sure.  You will have to change the threading stuff anyway and what's
> working in user space is pretty different from how you do it in Cygwin.
> Also, wouldn't it be nice to use the methods already provided by the OS
> for just such an opportunity?  Changing the pread/pwrite/prw_open
> methods to support aio_read/aio_write should be pretty straightforward.

Yes, I've got the idea now.  I'll go work on this direction and report 
back when I have something, modulo holidays and stuff.

Right now I have a patch to implement sigtimedwait() since it's similar to 
the already implemented sigwait() and sigwaitinfo().  I also have a patch 
that extends support for getting and setting cygthread and pthread names 
via cygwin_external().  I'll submit these two patches shortly.

Thanks for the patient internals explanations, as always.

..mark