[PATCH] manual: Drop incorrect statement on PIPE_BUF and blocking writes
Štěpán Němec
stepnem@smrk.net
Mon Mar 25 21:32:36 GMT 2024
On Mon, 25 Mar 2024 12:20:14 -0400
Zack Weinberg wrote:
> On Mon, Mar 25, 2024, at 8:13 AM, Štěpán Němec wrote:
>>>> Reading or writing a larger amount of data may not be atomic; for
>>>> example, output data from other processes sharing the descriptor may be
>>>> -interspersed. Also, once @code{PIPE_BUF} characters have been written,
>>>> -further writes will block until some characters are read.
>>>> +interspersed.
>>>
>>> Maybe “further may block” instead? I think the reference to PIPE_BUF
>>> and blocking could still be helpful, except that it's not a guarantee,
>>> as you correctly point out.
>
> It's not correct to say that a write of 65536 bytes will _never_
> block. Rather, the pipe capacity on Linux is (by default) 65536
> bytes, and, if nothing is reading, _any write_ that tries to put a
> 65537th byte into the pipe will block. For example, both of these
> will wait 1s before printing "all written":
>
> { dd if=/dev/zero bs=1 count=1 status=none;
> dd if=/dev/zero bs=65536 count=1 status=none;
> echo 'all written' >&2; } |
> { sleep 1; wc -c; }
>
> { dd if=/dev/zero bs=1 count=1 status=none;
> dd if=/dev/zero bs=65535 count=1 status=none;
> dd if=/dev/zero bs=1 count=1 status=none;
> echo 'all written' >&2; } |
> { sleep 1; wc -c; }
This seems correct and perhaps interesting, but how is it
relevant? I did not "say that a write of 65536 bytes will
_never_ block". I used a simple example to illustrate why
the statement in the manual about PIPE_BUF being the factor
causing blocking write was incorrect.
> I agree that it is weird to talk about this in a section that's
> nominally about atomicity. But I think we shouldn't be calling the
> "no interspersed data from other processes" behavior that we're trying
> to describe here "atomicity" at all!
Why? The very document you cite below (POSIX write(2))
makes the _atomic_ ("A write is atomic if the whole amount
written in one operation is not interleaved with data from
any other process. [...] This volume of POSIX.1-2017 does
not say whether write requests for more than {PIPE_BUF}
bytes are atomic, but requires that writes of {PIPE_BUF} or
fewer bytes shall be atomic.") vs _blocking_ ("The effective
size of a pipe or FIFO (the maximum amount that can be
written in one operation without blocking) may vary
dynamically, depending on the implementation, so it is not
possible to specify a fixed value for it.") distinction
right at the beginning of RATIONALE, not mentioning that the
terminology seems well established, and etymologically
fitting (ἄτομος meaning “indivisible”).
> Quoting
> <https://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html>:
>
> # Write requests to a pipe or FIFO shall be handled in the same way as
> # a regular file with the following exceptions:
> ...
> # * Write requests of {PIPE_BUF} bytes or less shall not be
> # interleaved with data from other processes doing writes on the
> # same pipe. Writes of greater than {PIPE_BUF} bytes may have data
> # interleaved, on arbitrary boundaries, with writes by other
> # processes, whether or not the O_NONBLOCK flag of the file status
> # flags is set.
>
> This is a weak statement.
How so? See here for the definition of "shall" in POSIXspeak:
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap01.html#tag_01_05_05
> It does *not* guarantee "that nothing else in the system
> can observe a state in which it is partially complete," as
> the manual currently puts it.
I admit I'm unable to extract much useful meaning from the
vague "nothing else in the system", but if we restrict our
perspective to the two ends of a pipe, "can[not] observe a
state in which it is partially complete" sounds about right,
doesn't it?
> Nor does it guarantee anything about how much a process
> reading from the pipe will receive if it does a larger
> read than the write. (To put that another way, if you
> write data packets to a pipe, the reader cannot use the
> return value of read() to tell how big the packets were.)
This seems to be confusing atomicity with blocking again.
> Also, it's not clear to me from what you wrote, whether Linux extends
> the no-interleaved-data guarantee writes larger than PIPE_BUF as long
> as they are smaller than the pipe capacity,
I don't know about any such guarantee.
> but if it does, we should say so only in a way that makes
> it clear it's not portable to rely on that.
>
> So I propose the appended revision to pipe.texi instead of what you
> proposed. It moves all this discussion to the beginning of the
> chapter and explains everything more thoroughly, and hopefully
> also correctly.
FWIW, I find your proposed text clear, helpful and matching
my understanding, and would welcome it to supersede my patch.
Thanks,
Štěpán
More information about the Libc-alpha
mailing list