fflush on closed std stream

[Eric Blake <ebb9 at byu dot net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

POSIX is clear that calling fclose(f), fflush(f) gives unspecified
behavior.  And in general, this is true, since FILE* are allocated by
f[d]open and fclose frees that memory.  But there is a special case of
std{in,out,err}, where the FILE object persists for the lifetime of the
program, even when the program is started with the underlying fd's
inherited as closed.

Also, POSIX states that fflush must set errno to EBADF if called on a
stream with an invalid fd, but after fclose (stdout), newlib currently
treats fflush(stdout) as success (although the fact that the program has
already done undefined behavior means that we can't claim it is wrong,
just poor QoI).

I recently ran into a problem where the m4 program, when run under cygwin
1.5, is able to successfully report fclose() failures inside an atexit()
handler.  But it is reporting that error via a generic error() method that
blindly calls fflush(stdout) prior to writing to stderr.  Fortunately,
most platforms tolerate this (particularly if you don't care about errno
or the return status after the failed fflush).  But when running the same
m4 test case under cygwin 1.7, and when coupled with the libsigsegv
library, the attempt to _flockfile the closed stdout results in a
segfault.  http://cygwin.com/ml/cygwin/2009-07/msg00655.html

This patch appears to fix the issue.  Is it safe to probe fp->_flags prior
to the _flockfile, in order to skip the lock if the stream is not
currently visiting a file?  Or do I have to approach this in some other
manner?

2009-07-18  Eric Blake  <ebb9@byu.net>

	Avoid a fault from locking a closed standard file.
	* libc/stdio/fflush.c (_fflush_r): Give up early if stream has
	been previously closed.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkphwQMACgkQ84KuGfSFAYAHBwCfV4GmUppGEXcINXXCC5g3qp0I
IQQAn0BZI9msKV9qbQd9WEhtKRLX8LIe
=L1za
-----END PGP SIGNATURE-----

Index: libc/stdio/fflush.c
===================================================================
RCS file: /cvs/src/src/newlib/libc/stdio/fflush.c,v
retrieving revision 1.13
diff -u -p -r1.13 fflush.c
--- libc/stdio/fflush.c	31 Oct 2008 21:08:03 -0000	1.13
+++ libc/stdio/fflush.c	18 Jul 2009 12:32:19 -0000
@@ -93,6 +93,12 @@ _DEFUN(_fflush_r, (ptr, fp),
 
   CHECK_INIT (ptr, fp);
 
+  if (!fp->_flags)
+    {
+      ptr->_errno = EBADF;
+      return EOF;
+    }
+
   _flockfile (fp);
 
   t = fp->_flags;