RE: tee piping to head gives error message

["Buchbinder, Barry (NIH/NIAID)" <BBuchbinder at niaid dot nih dot gov>]

At Monday, February 07, 2005 1:13 PM, Dave Korn wrote:
>> -----Original Message-----
>> From: cygwin-owner On Behalf Of Buchbinder, Barry (NIH/NIAID)
>> Sent: 07 February 2005 17:28
> 
>> As described in the Subject.  Though it seems that everything works
>> OK; tee just gives and error message under some circumstances.
>> 
>> /tmp> cat stafflist.htm | tee ttt | head > /dev/null
>> tee: standard output: Broken pipe
> 
>> So the file that tee write looks good and the write error is
>> only to the pipe.
> 
>   That'll be because head closed the pipe, and tee was still trying
> to write into it.

Further experimentation supported that.  Inserting cat between tee and head
did not eliminate the error message, while inserting sort did.
 
>> So head seems to be passing on the number of lines that ones
>> asks it to pass on.
> 
>   Given that the purpose of head is to print the first few lines of a
> file, it kind of makes sense to me that it would close the file after
> it's read them rather than keeping the input file open and manually
> reading-and-discarding the entire rest of it for no good reason.

Agreed.

>   So I reckon this is as-expected and by-design behaviour.

I might put it as "as-designed" rather than "by-design".  And for me, it
certainly was unexpected.  tee and head are both part of coreutils.  I would
expect that all coreutils would behave the same for head closing the pipe,
but they don't.  And I would also expect that all utilities in a package
that includes a utility that breaks pipes as a normal course of its
operation would be silent when the two utilities are used together.  I would
expect that tee pipes to head more often than something nasty happens and a
pipe just breaks.

This seems like something the coreutils maintainer might want to address
with the upstream maintainers, or to patch himself.  (I won't complain if he
doesn't patch it.  Taking on coreutils was quite a commitment -- well
deserving of the two gold stars -- and I know that fixing this may be a low
priority.)  Unfortunately, though PTC, I'm not capable of providing a patch.
In any case, tee seems to save its input as desired, so while the error
message is annoying and misleading, I suppose that one can live with it.

>> So it seems to be specific to piping from tee to head.
> 
>   It's just that tee notices when a write to stdout fails, whereas
> most applications are more loosely coded and don't check.
> 
>> But the number of lines/bytes at which the error disappears
>> does not seem to be constant.
> 
>   Umm, no, .... it's equal to the number of lines in the source file.

No.  It should be equal to the numbers of lines in the source file but is
not.  The error message went away around 126 or 130 lines, while the source
file had 556.

(I would speculate that the disappearance of the error messages when enough
lines are provided might have something to do with buffering, but I'm not a
programmer and speculation by we mere "users" is sometimes discouraged.  As
for why it is not consistent ...)
 
>> So it seems that when given enough lines/bytes, there is no
>> error message.
> 
>   If you ask head for less lines than are in the source file, it
> closes its input (which is tee's output) before tee has written the
> entire file, and tee complains.  OTOH if you ask head for >= the
> number of lines in the source file, tee completes, and head notices
> that it has no more input and exits gracefully. 
> 
>> Everything should be up to date.
> 
>   Except perhaps your caffeine levels this morning ?
> 
>     cheers,
>       DaveK


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/