The vdprintf_chk() in debug/vdprintf_chk.c does not return EOF when writing to a closed file. It instead returns the number of characters that would have been written.
Comparing the code in vdprintf_chk() with that in _IO_vdprintf() (libio/iovdprintf.c), there are two lines in _IO_vdprintf() checking for an EOF and adjusting the return value that are missing from vdprintf_chk() resulting in this issue. The rest of the code in the two functions is almost identical.
Adding the missing lines to vdprintf_chk() fixes the issue.
The two lines that are missing are:
*** vdprintf_chk.c.orig 2016-06-09 07:40:16.038497365 -0700
--- vdprintf_chk.c 2016-06-09 07:40:46.697971100 -0700
*** 60,65 ****
--- 60,68 ----
done = _IO_vfprintf (&tmpfil.file, format, arg);
+ if (done != EOF && _IO_do_flush (&tmpfil.file) == EOF)
+ done = EOF;
The vdprintf_chk does indeed return EOF when trying to write on a closed file. It will currently call:
And the __lseek64 will return -1/EBADF and thus _IO_new_file_attach will fail.
The code difference with default vdprintf is, in fact, BZ#11319, where vdprintf_chk does not return an error output error is encountered (the same example reported on BZ#11319 fails with -D_FORTIFY_SOURCE=2 -O2).
I will close this as invalid and re-open BZ#11319.