Re: [Bug libc/6050] iconv(1) buffers all of stdin in memory

[OndÅej BÃlka <neleai at seznam dot cz>]

On Sat, Aug 10, 2013 at 04:01:22PM +0000, bugdal at aerifal dot cx wrote:
> http://sourceware.org/bugzilla/show_bug.cgi?id=6050
> 
> Rich Felker <bugdal at aerifal dot cx> changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |bugdal at aerifal dot cx
> 
> --- Comment #1 from Rich Felker <bugdal at aerifal dot cx> ---
> Ping. This bug still exists. I traced it to a comment in the source:
> 
> /* we have a problem with reading from a desriptor since we must not
>    provide the iconv() function an incomplete character or shift
>    sequence at the end of the buffer.  Since we have to deal with
>    arbitrary encodings we must read the whole text in a buffer and
>    process it in one step.  */
> 
> See
> http://sourceware.org/git/?p=glibc.git;a=blob;f=iconv/iconv_prog.c;h=1a1d0d0cf45c0d747a8090bc234addd9e49f1ba7;hb=HEAD#l561
> 
> The claims made in the comment are simply erroneous. Per POSIX, the iconv
> function returns (size_t)-1 with errno set to EINVAL to indicate "Input
> conversion stopped due to an incomplete character or shift sequence at the end
> of the input buffer." This is a different condition from EILSEQ, and thus the
> caller can detect and recover from it simply by moving the remaining bytes of
> the input buffer to the beginning, re-filling the buffer, and calling iconv
> again.
>
This would work only for stateless encodings. You cannot do this with
ISO-2022-JP as you would need additional argument to save state.