Created attachment 12566 [details] simple test program It seems that according to RFC 3629, -1 should be returned here since an invalid character is encoded, that's U+2f43580, not in range [U+0, U+10ffff]. This came up because Python does this conversion using mbstowcs() and/or mbrtowc(), but then later goes to check that valid characters were returned. The python discussion is here: https://bugs.python.org/issue35883 but given the language in RFC 3629, it seems like an issue in glibc: 3. UTF-8 definition [...] In UTF-8, characters from the U+0000..U+10FFFF range (the UTF-16 accessible range) are encoded using sequences of 1 to 4 octets. [...] (hexadecimal) | (binary) --------------------+--------------------------------------------- 0000 0000-0000 007F | 0xxxxxxx 0000 0080-0000 07FF | 110xxxxx 10xxxxxx 0000 0800-0000 FFFF | 1110xxxx 10xxxxxx 10xxxxxx 0001 0000-0010 FFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx [...] Implementations of the decoding algorithm above MUST protect against decoding invalid sequences. [...]
Probably the same issue as bug 2373.
Hm, yeah, that sounds the same, I had only searched for the specific function(s), not the broader issue. I guess I won't hold my breath for this to get fixed then ...
Marking as duplicate per comment 2. *** This bug has been marked as a duplicate of bug 2373 ***
Hm, note though. I was just mentioning this to somebody, and 2373 talks about *encoding* while this is mostly about *decoding*. So it's related, but not exactly the same. Up to you whether or not you want to treat it as a duplicate, but it's two sides of the same coin. An argument could be made, for example, for allowing *encoding* it (since why did the application store something >0x10ffff in a wchar_t to start with, that was already invalid) but not *decoding* it, even if that breaks the round-trip property.
Fair point. I have retitled bug 2373 accordingly.