The bug probably happens because open_memstream() does not initialize the _IO_wide_data struct member or the pointer to it. In many cases this bug is not triggered because malloc() returns zeroed memory. But you can reliably crash glibc (both Debian stable’s version and current git master) with this program:
int main(int argc, char **argv)
fp = open_memstream(&stream, &slen);
Run the program with env MALLOC_PERTURB_=170.
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7ac4050 in _IO_getwc (fp=0x601010) at getwc.c:42
42 getwc.c: No such file or directory.
(gdb) bt full
#0 0x00007ffff7ac4050 in _IO_getwc (fp=0x601010) at getwc.c:42
result = <optimized out>
#1 0x000000000040058e in main (argc=1, argv=0x7fffffffe578) at t.c:12
stream = 0x7fffffffe570 "\001"
slen = 4195408
fp = 0x601010
Some system information:
OS/Kernel: Debian 7.0 / Linux 3.2.0-4-amd64 x86_64
glibc version: 2.13-38 (Debian) / glibc-2.17-543-gccdad15 (git master)
This patch fixes the bug. (It might be more reasonable, however, to include this line somewhere in the _IO_no_init() function, not sure.)
diff --git i/libio/memstream.c w/libio/memstream.c
index 34534e2..768954c 100644
@@ -89,6 +89,7 @@ open_memstream (bufloc, sizeloc)
_IO_JUMPS ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf) = &_IO_mem_jumps;
_IO_str_init_static_internal (&new_f->fp._sf, buf, _IO_BUFSIZ, buf);
new_f->fp._sf._sbf._f._flags &= ~_IO_USER_BUF;
+ new_f->fp._sf._sbf._f._wide_data = NULL;
new_f->fp._sf._s._allocate_buffer = (_IO_alloc_type) malloc;
new_f->fp._sf._s._free_buffer = (_IO_free_type) free;
(The crash was discovered by Vitezslav Cizek, see https://bugzilla.novell.com/show_bug.cgi?id=813498 )
Not exactly a bug. The stream returned by open_memstream is byte-oriented, so calling any wide character stdio function on it invokes undefined behavior. Similarly, open_wmemstream returns a wide-oriented stream, so calling any byte-oriented stdio function on it would invoke undefined behavior.
True. When you issue fwide() on such a stream, it is already determined that it is char-oriented. I didn’t see that until now.
In my opinion it would be good to expose this “undefined” behavior by invalidating the pointer. Because right now you’ll only see this bug in “lucky” circumstances; often it will go unnoticed.
Fixed on master by commit bae143d2702e5ca1265c55b06072afba01bfc07a