Created attachment 8869 [details]
ftell returns wrong result and then crashes

There are several problems when ftell is called for a stream and the following conditions are met:
- the stream is wide-oriented,
- it has unflushed data,
- the current locale uses a variable-length encoding.

The problems are illustrated by the attached sample program and are described below. Quick-n-dirty patch is attached too.

When the conditions are met, at some point, the execution is passed to this fragment:

----------------------------------------------------------------------
https://sourceware.org/git/?p=glibc.git;a=blob;f=libio/wfileops.c;h=99f9c8fe65726ac4c163c7d61bc7ee58e05e986e;hb=HEAD#l697

 697             {
 698               size_t delta = (fp->_wide_data->_IO_write_ptr
 699                               - fp->_wide_data->_IO_write_base);

Apparently, this is the number of wide chars not yet written to the stream.

 700 
 701               /* Allocate enough space for the conversion.  */
 702               size_t outsize = delta * sizeof (wchar_t);

Problem 1: to get the size of output buffer you have to multiply the number of chars by something like MB_LEN_MAX instead of sizeof (wchar_t). If there is not enough space conversion will fail.

 703               char *out = malloc (outsize);

Problem 2: the result of malloc is checked. In out of memory condition the program will crash.

 704               char *outstop = out;
 705               const wchar_t *in = fp->_wide_data->_IO_write_base;
 706 
 707               enum __codecvt_result status;
 708 
 709               __mbstate_t state = fp->_wide_data->_IO_last_state;
 710               status = (*cv->__codecvt_do_out) (cv, &state,
 711                                                 in, in + delta, &in,
 712                                                 out, out + outsize, &outstop);
 713 
 714               /* We don't check for __codecvt_partial because it can be
 715                  returned on one of two conditions: either the output
 716                  buffer is full or the input sequence is incomplete.  We
 717                  take care to allocate enough buffer and our input
 718                  sequences must be complete since they are accepted as
 719                  wchar_t; if not, then that is an error.  */
 720               if (__glibc_unlikely (status != __codecvt_ok))
 721                 {
 722                   free (out);
 723                   return WEOF;

Problem 3: WEOF is invalid error code for ftell. It prints 4294967295 on my x86-64 machine.

 724                 }
 725 
 726               offset += outstop - out;
 727               free (out);
 728             }

----------------------------------------------------------------------

Introduced in https://sourceware.org/git/?p=glibc.git;a=commit;h=000232b9bcbf194f1e5fd0ff380000f341505405 .