fcntl with O_APPEND fails to force append mode on stderr for native Windows programs

[Tom Honermann <thomas dot honermann at synopsys dot com>]

This is a follow up to a bug originally reported as https://cygwin.com/ml/cygwin/2015-05/msg00140.html.  We continue to face this issue and recently spent some time trying to further isolate it.  We now have a better reproducer (below) that does not require GNU make.

>From the original bug report, we bisected changes to GNU make and eventually identified that the reported problem was introduced by this change:

  *   http://git.savannah.gnu.org/cgit/make.git/commit/?id=9453a7451d66ee1ea5762a75161bf8a818c01a1f

The following program demonstrates the problem.

$ cat test.c
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

void set_append_mode(int fd) {
  int flags = fcntl(fd, F_GETFL, 0);
  if (flags >= 0)
    fcntl(fd, F_SETFL, flags | O_APPEND);
}

int main() {
  set_append_mode(fileno(stdout));
  set_append_mode(fileno(stderr));
  system("echo first > out");
  system("echo second >> out");
  system("net 2>> out");
}

The expected output of this program is that file 'out' have the content "first", followed by "second", followed by the error message emitted by the Windows 'net' utility (any native Windows program that writes to stderr should suffice to reproduce this issue).  And indeed, that is what is seen when the program is compiled and run with stdout and stderr set to the console.

$ gcc test.c -o test
$ ./test
$ cat out
first
second
The syntax of this command is:

NET
    [ ACCOUNTS | COMPUTER | CONFIG | CONTINUE | FILE | GROUP | HELP |
      HELPMSG | LOCALGROUP | PAUSE | SESSION | SHARE | START |
      STATISTICS | STOP | TIME | USE | USER | VIEW ]

Here is where things go bananas.  If the program is run with stdout initially redirected to a pipe, then the stderr output from the native Windows program ends up overwriting the stdout output from the two calls to echo:

$ ./test | cat
$ cat out
The syntax of this command is:

NET
    [ ACCOUNTS | COMPUTER | CONFIG | CONTINUE | FILE | GROUP | HELP |
      HELPMSG | LOCALGROUP | PAUSE | SESSION | SHARE | START |
      STATISTICS | STOP | TIME | USE | USER | VIEW ]

Note that the lines containing "first" and "second" are now absent in the file.

If the calls to set_append_mode are commented out, then the output is consistent in both cases.

The problem only occurs with native Windows programs that write to stderr.  If the above test is changed to append the stdout output from a native Windows program, then the output is consistent.

I've reproduced this problem on ancient versions of Cygwin such as 1.7.23 and recent versions such as 2.11.2.

Tom.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple