Re: codeset problems in wprintf and wcsftime

[Andy Koppe <andy dot koppe at gmail dot com>]

Corinna Vinschen:
> while working on finalizing locale support for Cygwin it suddenly
> occured to me that we have a problem in wprintf and wcsftime.
>
> Let's assume a funny combination of localization variables in the user's
> environment:
>
> ÂLANG=de_DE.utf8
> ÂLC_TIME=ja_JP.eucjp
> ÂLC_NUMERIC=en_US.iso88591
>
> Yes, it's pretty unlikely, but nevertheless possible and valid.
>
> So, at setlocale time we read and store the localized strings in the
> codeset specified by the localization variable:
>
> Â- __locale_charset() Â Â Â Â Â Â returns UTF-8
> Â- __get_current_time_locale() Â Âreturns data stored in EUC-JP
> Â- __get_current_numeric_locale() returns data stored in ISO-8859-1
> Â- localeconv() Â Â Â Â Â Â Â Â Â returns with decimal_point and
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â thousands_sep stored in ISO-8859-1,
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â and all other strings from the
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â LC_MONETARY category in UTF-8.
> Â- nl_langinfo() Â Â Â Â Â Â Â Â ÂCODESET is UTF-8,
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â strings from the LC_TIME category are
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â returned in EUC-JP,
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â strings from LC_MESSAGES are returned
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â in UTF-8
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â RADIXCHAR and THOUSEP are returned in
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ISO-8859-1.
>
> This is no problem at all as long as you call the multibyte variations
> printf and strftime, the user gets what she asked for, and who are we
> to ask the user for the reason behind this choice.

Have you verified that the user does indeed get a mix of charsets when
doing this on glibc?

I'm asking because another alternative to the solutions you outlined
might be to store those strings as wchar versions only, to be used
directly in wprintf and converted to the LC_CTYPE character set when
needed in printf. That way, the user would always get readable output.


> - Store the charset not only for LC_CTYPE, but for each localization
> Âcategory, and provide a function to request the charset.
> ÂThis also requires to store the associated multibyte to widechar
> Âconversion functions, obviously, and to call the correct functions
> Âfrom wprintf and wcftime.
>
> - Redefine the locale data structs so that they contain multibyte and
> Âwidechar representations of all strings. ÂUse the multibyte strings
> Âin the multibyte functions, the widechar strings in the widechar
> Âfunctions.
>
> Personally I'd prefer the second approach.

Agreed. Sounds like less overhead.

Andy