Clearing O_NONBLOCK from a pipe may lose data

[Thomas Wolff]

Am 20.02.2015 um 11:13 schrieb Corinna Vinschen:
> On Feb 20 08:56, Thomas Wolff wrote:
>> Am 20.02.2015 um 00:47 schrieb Andrey Repin:
>>> Greetings, Thomas Wolff!
>>>
>>>> Am 19.02.2015 um 10:51 schrieb Corinna Vinschen:
>>>>> On Feb 18 22:08, Lasse Collin wrote:
>>>>>> (Please Cc me when replying, I'm not subscribed to the list.)
>>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> I suspect that there is a bug in Cygwin:
>>>>>>
>>>>>> 1. Create a pipe with both ends in blocking mode (O_NONBLOCK
>>>>>>      is not set).
>>>>>> 2. The writer sets its end to non-blocking mode.
>>>>>> 3. The writer writes to the pipe.
>>>>>> 4. The writer restores its end of the pipe to blocking mode
>>>>>>      before the reader has read anything from the pipe.
>>>>>> 5. The writer closes its end of the pipe.
>>>>>> 6. The reader reads from the pipe in blocking mode. The last
>>>>>>      bytes written by the writer never appear at the reader,
>>>>>>      thus data is silently lost.
>>>>>>
>>>>>> Omitting the step 4 above makes the problem go away.
>>>>> I can imagine.  A few years back, when changing the pipe code to
>>>>> using overlapped IO, we stumbled over a problem in Windows.  When
>>>>> closing an overlapped pipe while I/O is still ongoing, Windows
>>>>> simply destroys the pipe buffers without flushing the data to the
>>>>> reader.  This is not much of a problem for blocking IO, but it
>>>>> obviously is for non-blocking.
>>>>>
>>>>> The workaround for this behaviour is this:  If the pipe is closed, and
>>>>> this is the writing side of a nonblocking pipe, a background thread gets
>>>>> started which keeps the overlapped structure open and continues to wait
>>>>> for IO completion (i.e. the data has been sent to the reader).
>>>>>
>>>>> However, if you switch back to blocking before closing the pipe, the
>>>>> aforementioned mechanism does not kick in.
>>>> Could not "switching back to blocking" simply be handled like closing as
>>>> far as the waiting is concerned,
>>>> thus effectively flushing the pipe buffer?
>>> You can't "just flush" it, if the receiving end isn't reading from it.
>> By flushing I meant actually waiting until it's been consumed at the
>> other end in this case, if that's technically feasible.
> You mean the actual act of changing the descriptor from non-blocking
> to blocking, as in fcntl(fd, F_SETFL), shall perform the same action
> of waiting as the close call on non-blocking descriptors does?
Yes.
>> I see no strict requirement that the fcntl call removing O_NONBLOCK from
>> a file descriptor should itself still be handled as nonblocking (it can
>> well be argued that the flag is changed first and then the call is
>> allowed to block) - and even if this were not proper it is certainly
>> more acceptable than losing data.
> I'm not sure that works as desired, but it's probably worth a try.  An
> fcntl method for pipes has to be added (there is none yet, it's all done
> in fhandler_base::fcntl), then the F_SETFL command would have to be
> augmented to create a thread calling FlushFileBuffers (which is
> *supposed* to work on pipe handles but I never tried it myself), and the
> fcntl call would have to wait for thread completion, allowing
> interruption by signals (calling cygwait, that is).
where the actual code (as I understood you) could be copied from close() 
code. Although I looked into the code and didn't find the place where 
close would lead to FlushFileBuffers...
> The question with stuff like this is usually, how long are you willing to wait?
Indefinitely...
>    You never know what the reader side of a pipe is doing.  It
> might just be busy and intends to read from the pipe in a second, a
> minute, or an hour.
... like it is normal when feeding a pipe in blocking mode (which we 
just switched to). I don't see a problem here.
------
Thomas

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple