Large file iostreams

[Martin Sebor]

Pétur Runólfsson wrote:

> Nathan Myers wrote:
> 
...
>   state initialized, if at the beginning of a sequence
> 
> But it is never specified what is meant by "initialized".
> I agree that this seems to be a real problem when using codecvt
> directly.

Not really. Portable code just needs to make sure to zero it out
before passing it to codecvt. Footnote 226 in 22.2.1.5.2, p5 even
suggests how it should be done in the generic case -- by assigning
it stateT().

> 
> However, I don't think this matters for basic_filebuf:
> The only functions that seem to need to know what the
> initial state is supposed to be are basic_filebuf::open() and
> basic_filebuf::seekoff(). These are both provided by the
> implementation, and implementors presumably know how to
> intitialize mbstate_t (since it's an implementation-defined
> type).

Yes. And C says it must be zeroed out, e.g., by assigning it
the value of mbstate_t() or calling memset(0) on it.

> 
> There is no need for the standard to say how to initialize other
> state_types, since the use of those with basic_filebuf is
> implementation defined.

Presumably stateT ()? :)

> 
> 
>>Without it, all the multibyte stuff is
>>meaningless.
> 
> 
> Not really. It is possible to write codecvt facets for utf-8
> and other stateless encodings without ever using the conversion
> state.
> 
> There doesn't seem to be any real requirement that stateful
> encodings should be supported. Support for stateful encodings
> adds much complexity to basic_filebuf, so this may be by design.

Getting it right is a little tricky but it doesn't make filebuf
that much more complex. Our implementation works with stateful
encodings such as ISO 2022 and our fstream header (including the
.cc file) is only 150 lines longer than gcc 3.3's fstream
(including the .tcc) file. What does become complex is codecvt
which cannot be portably implemented in terms of the C library
anymore. The code that implements a small subset of ISO 2022
(JIS X 0208-1978, JIS X 0208-1983, JIS X 0212-1990, GB 2312-80,
and KSC 5601) in our library is over 2KLOC.

Martin