is the concept of codecvt_t::partial broken ?

[Oncaphillis]

Thanks for the fast reply
On 09/26/2012 12:30 PM, Paolo Carlini wrote:
> Hi,
>
> On 09/26/2012 12:18 PM, Oncaphillis wrote:
>> Hi,
>> I'm writing yet another char encoding translator with the help of 
>> codecvt< >
>> and stumbled across the following problem. When I try to interprese a 
>> byte sequence
>> as ShiftJis I never get the result code codcvt_t::partial even if I 
>> feed a single
>> 0xe0 as the only input which the iconv should interprete as an 
>> incomplete two byte
>> sequence. The same seems to hold true with other multi char encodings 
>> as well
> now I don't have the time to analyze the details of this (and it would 
> be very difficult without a complete self-contained testcase) but in 
> general the current codecvt code works pretty well for us. I would 
> suggest: 1- Have a look to the existing testcases in our testsuite, 
> which all pass, and see if you can learn something from those about 
> your specific case; 2- We may have issues with encoding -1 locales, 
> thus I would at first try >= 0 locales. ja_JP.sjis is -1, right?
>
  Please find attached code which hopefully compiles cleanly on any system.
It tries to transform a given argument string into a std::wstring assuming
an encoding given by the optinal second arg which defaults to en_US.utf8.
The transcoding is done one input char after the other and the current input
and output is printed out.Since it's a profane utf8->wchar_t transcoding
I'm pretty sure it's me who doesn't understand the concept.

On my system with utf8 locale

./a.out 'Hellö Wörld' en_US.utf8

gives me:

<0x48> [0x0048]  // 'H'
<0x65> [0x0065]  // 'e'
<0x6c> [0x006c]  // 'l'
<0x6c> [0x006c]  // 'l'
<0xc3> []        // start of the german umlaut ö-- here I would expect 
to get a codecvt::partial result
<0xb6> [0x00f6]  // second part of the german umlaut ö
<0x20> [0x0020]  // etc....
<0x57> [0x0057]
<0xc3> []
<0xb6> [0x00f6]
<0x72> [0x0072]
<0x6c> [0x006c]
<0x64> [0x0064]


Thanks
Sebastian

> Thanks,
> Paolo.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: strtools.cpp
Type: text/x-c++src
Size: 2127 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/libstdc++/attachments/20120926/5313f2d3/attachment.bin>