RFC: locale-source validation script

Zack Weinberg zackw@panix.com
Wed Jul 26 01:11:00 GMT 2017 

In the thread about using fewer <Uxxxx> escapes in the locale source
files, Carlos was concerned that, if we went over to UTF-8 for
everything, not just decoded the escapes that represent ASCII, it
would be easy for people to miss incorrectly encoded character
sequences - text that isn't normalized, for instance, or homograph
characters that *look* OK but are incorrect for the locale's language.

It seems to me that this sort of check is not something that humans
should have to do by eye; rather, it's a job for a linter.  So I wrote
one. :)  It currently looks for "inappropriate" escape sequences and
characters, using a quite strict notion of "inappropriate"; for
strings that are not in Unicode Normalization Form C; and for strings
that cannot be transcoded to the legacy charset for the locale (as
defined by a "% Charset: xxx" annotation in the file - note that not
all the files have such annotations).

It is not ready for prime time; it is very slow (Python isn't really
designed to go character-by-character through a file; it can probably
be sped up with a cleverer lexer) and it finds a whole bunch of
existing errors, some of which may not actually be _problems_, if you
see what I mean.  I've attached the script and the result of running
it over all of the files in localedata/locales/.  But it's ready for
people to poke at.

Some notes on what I found with it:

 - Many of the existing locale files have non-ASCII text in their
comments.  This text is _invariably_ encoded in UTF-8.  That no one
has complained about this is a weak argument in favor of it being safe
to go ahead with UTF-8 - there might be localedef implementations that
accept non-ASCII in comments but not elsewhere, I suppose.

 - A few of the existing locale files have non-ASCII text in strings
already.  Again, this is invariably encoded in UTF-8, and I think so
far it's limited to LC_IDENTIFICATION (accented characters in the
author's name, that sort of thing).

 - Quite a few of the existing locale files have strings outside
LC_IDENTIFICATION that contain "raw" ASCII already.  (This is why I
had to write a full-on lexer for the format; existing files contain
both % inside "" strings, and " characters inside % comments.  That
was a step beyond what I felt like doing with regexes.)

- There are quite a few strings that aren't NFC and I suspect it's
going to take expert knowledge of the languages involved to tell if
that's desirable.

- A significantly cleverer homograph checker is wanted, one that keys
off of the ISO language code, rather than the legacy charset.  (The
legacy-charset check is already done by localedef, AFAIK, and
localedef has more complete information when it does that.)

- The complaints about "inappropriate character '\t'" are all caused
by _unintentional_ tabs inside strings.  If you write

message "xyz/
         abc"

the whitespace on the second line gets included in the string, which
is not what you want.  The linter currently only detects this when
that indentation is done with tabs, but I think it should probably
detect spaces as well.  If you _mean_ to put a tab in a string write
<U0009>. :-)

- All of the complaints about "inappropriate escape sequences" boil
down to people forgetting that / is an escape character in these
files, and writing strings with slashes in them.  This is limited to
LC_IDENTIFICATION, so it's cosmetic, but it's still wrong and IMNSHO
justifies the linter insisting that you're only supposed to use / to
escape <>/".

- Speaking of, why is it that every single locale source file uses %
for comments and / for escapes, instead of the default # for comments
and \ for escapes?  It seems gratuitous and it made the linter harder
to write.

 - Suggestions for additional checks are welcome.

zw
-------------- next part --------------
localedata/locales/aa_DJ:31: inappropriate escape sequence '/L'
localedata/locales/aa_ER:31: inappropriate escape sequence '/L'
localedata/locales/aa_ET:31: inappropriate escape sequence '/C'
localedata/locales/agr_PE:34: inappropriate character 'ú' in string
localedata/locales/ak_GH:30: inappropriate escape sequence '/ '
localedata/locales/an_ES:32: inappropriate character 'é' in string
localedata/locales/as_IN:140: string not normalized:
  source: 09B9 09DF
     nfc: 09B9 09AF 09BC
localedata/locales/as_IN:141: string not normalized:
  source: 09A8 09B9 09DF
     nfc: 09A8 09B9 09AF 09BC
localedata/locales/bn_BD:14: inappropriate escape sequence '/B'
localedata/locales/bn_IN:112: string not normalized:
  source: 099C 09BE 09A8 09C1 09DF 09BE 09B0 09BF
     nfc: 099C 09BE 09A8 09C1 09AF 09BC 09BE 09B0 09BF
localedata/locales/bn_IN:113: string not normalized:
  source: 09AB 09C7 09AC 09CD 09B0 09C1 09DF 09BE 09B0 09BF
     nfc: 09AB 09C7 09AC 09CD 09B0 09C1 09AF 09BC 09BE 09B0 09BF
localedata/locales/da_DK:145: string not normalized:
  source: 0041 0308
     nfc: 00C4
localedata/locales/da_DK:147: string not normalized:
  source: 0041 030A
     nfc: 00C5
localedata/locales/da_DK:151: string not normalized:
  source: 004F 0308
     nfc: 00D6
localedata/locales/da_DK:155: string not normalized:
  source: 0061 0308
     nfc: 00E4
localedata/locales/da_DK:157: string not normalized:
  source: 0061 030A
     nfc: 00E5
localedata/locales/da_DK:161: string not normalized:
  source: 006F 0308
     nfc: 00F6
localedata/locales/de_DE:50: string not normalized:
  source: 0041 0308
     nfc: 00C4
localedata/locales/de_DE:52: string not normalized:
  source: 004F 0308
     nfc: 00D6
localedata/locales/de_DE:54: string not normalized:
  source: 0055 0308
     nfc: 00DC
localedata/locales/de_DE:56: string not normalized:
  source: 0061 0308
     nfc: 00E4
localedata/locales/de_DE:58: string not normalized:
  source: 006F 0308
     nfc: 00F6
localedata/locales/de_DE:60: string not normalized:
  source: 0075 0308
     nfc: 00FC
localedata/locales/de_DE:64: string not normalized:
  source: 0041 030A
     nfc: 00C5
localedata/locales/de_DE:66: string not normalized:
  source: 0061 030A
     nfc: 00E5
localedata/locales/de_IT:26: inappropriate escape sequence '/B'
localedata/locales/es_CU:28: inappropriate escape sequence '/2'
localedata/locales/et_EE:2233: inappropriate character '\t' in string
localedata/locales/et_EE:2233: inappropriate character '\t' in string
localedata/locales/et_EE:2234: inappropriate character '\t' in string
localedata/locales/et_EE:2234: inappropriate character '\t' in string
localedata/locales/et_EE:2235: inappropriate character '\t' in string
localedata/locales/et_EE:2235: inappropriate character '\t' in string
localedata/locales/et_EE:2236: inappropriate character '\t' in string
localedata/locales/et_EE:2236: inappropriate character '\t' in string
localedata/locales/et_EE:2237: inappropriate character '\t' in string
localedata/locales/et_EE:2237: inappropriate character '\t' in string
localedata/locales/et_EE:2238: inappropriate character '\t' in string
localedata/locales/et_EE:2238: inappropriate character '\t' in string
localedata/locales/fa_IR:107: string not normalized:
  source: 0627 0653
     nfc: 0622
localedata/locales/fa_IR:108: string not normalized:
  source: 0627 0654
     nfc: 0623
localedata/locales/fa_IR:109: string not normalized:
  source: 0627 0655
     nfc: 0625
localedata/locales/fa_IR:110: string not normalized:
  source: 0648 0654
     nfc: 0624
localedata/locales/fa_IR:112: string not normalized:
  source: 064A 0654
     nfc: 0626
localedata/locales/gd_GB:23: inappropriate escape sequence '/2'
localedata/locales/gl_ES:30: inappropriate character 'á' in string
localedata/locales/gl_ES:30: inappropriate escape sequence '/n'
localedata/locales/gv_GB:31: inappropriate character 'í' in string
localedata/locales/gv_GB:31: inappropriate character 'á' in string
localedata/locales/he_IL:33: inappropriate escape sequence '/r'
localedata/locales/he_IL:33: inappropriate escape sequence '/b'
localedata/locales/hif_FJ:115: inappropriate character '\t' in string
localedata/locales/hif_FJ:170: inappropriate character '\t' in string
localedata/locales/hr_HR:29: inappropriate escape sequence '/M'
localedata/locales/ht_HT:32: inappropriate escape sequence '/ '
localedata/locales/hu_HU:461: string not normalized:
  source: 0041 0301
     nfc: 00C1
localedata/locales/hu_HU:462: string not normalized:
  source: 0045 0301
     nfc: 00C9
localedata/locales/hu_HU:463: string not normalized:
  source: 0049 0301
     nfc: 00CD
localedata/locales/hu_HU:464: string not normalized:
  source: 004F 0301
     nfc: 00D3
localedata/locales/hu_HU:465: string not normalized:
  source: 004F 0308
     nfc: 00D6
localedata/locales/hu_HU:466: string not normalized:
  source: 004F 030B
     nfc: 0150
localedata/locales/hu_HU:467: string not normalized:
  source: 0055 0301
     nfc: 00DA
localedata/locales/hu_HU:468: string not normalized:
  source: 0055 0308
     nfc: 00DC
localedata/locales/hu_HU:469: string not normalized:
  source: 0055 030B
     nfc: 0170
localedata/locales/hu_HU:471: string not normalized:
  source: 0061 0301
     nfc: 00E1
localedata/locales/hu_HU:472: string not normalized:
  source: 0065 0301
     nfc: 00E9
localedata/locales/hu_HU:473: string not normalized:
  source: 0069 0301
     nfc: 00ED
localedata/locales/hu_HU:474: string not normalized:
  source: 006F 0301
     nfc: 00F3
localedata/locales/hu_HU:475: string not normalized:
  source: 006F 0308
     nfc: 00F6
localedata/locales/hu_HU:476: string not normalized:
  source: 006F 030B
     nfc: 0151
localedata/locales/hu_HU:477: string not normalized:
  source: 0075 0301
     nfc: 00FA
localedata/locales/hu_HU:478: string not normalized:
  source: 0075 0308
     nfc: 00FC
localedata/locales/hu_HU:479: string not normalized:
  source: 0075 030B
     nfc: 0171
localedata/locales/hy_AM:15: inappropriate escape sequence '/a'
localedata/locales/ig_NG:104: string not normalized:
  source: 0049 0323
     nfc: 1ECA
localedata/locales/ig_NG:105: string not normalized:
  source: 0069 0323
     nfc: 1ECB
localedata/locales/ig_NG:106: string not normalized:
  source: 0049 0323 0301
     nfc: 1ECA 0301
localedata/locales/ig_NG:107: string not normalized:
  source: 0069 0323 0301
     nfc: 1ECB 0301
localedata/locales/ig_NG:108: string not normalized:
  source: 0049 0323 0300
     nfc: 1ECA 0300
localedata/locales/ig_NG:109: string not normalized:
  source: 0069 0323 0300
     nfc: 1ECB 0300
localedata/locales/ig_NG:115: string not normalized:
  source: 004F 0323
     nfc: 1ECC
localedata/locales/ig_NG:116: string not normalized:
  source: 006F 0323
     nfc: 1ECD
localedata/locales/ig_NG:117: string not normalized:
  source: 004F 0323 0301
     nfc: 1ECC 0301
localedata/locales/ig_NG:118: string not normalized:
  source: 006F 0323 0301
     nfc: 1ECD 0301
localedata/locales/ig_NG:119: string not normalized:
  source: 004F 0323 0300
     nfc: 1ECC 0300
localedata/locales/ig_NG:120: string not normalized:
  source: 006F 0323 0300
     nfc: 1ECD 0300
localedata/locales/ig_NG:131: string not normalized:
  source: 0055 0323
     nfc: 1EE4
localedata/locales/ig_NG:132: string not normalized:
  source: 0075 0323
     nfc: 1EE5
localedata/locales/ig_NG:133: string not normalized:
  source: 0055 0323 0301
     nfc: 1EE4 0301
localedata/locales/ig_NG:134: string not normalized:
  source: 0075 0323 0301
     nfc: 1EE5 0301
localedata/locales/ig_NG:135: string not normalized:
  source: 0075 0323 0300
     nfc: 1EE5 0300
localedata/locales/ig_NG:136: string not normalized:
  source: 0055 0323 0300
     nfc: 1EE4 0300
localedata/locales/ig_NG:142: string not normalized:
  source: 004E 0307
     nfc: 1E44
localedata/locales/ig_NG:143: string not normalized:
  source: 006E 0307
     nfc: 1E45
localedata/locales/ig_NG:145: string not normalized:
  source: 0041 0301
     nfc: 00C1
localedata/locales/ig_NG:146: string not normalized:
  source: 0061 0301
     nfc: 00E1
localedata/locales/ig_NG:148: string not normalized:
  source: 0045 0301
     nfc: 00C9
localedata/locales/ig_NG:149: string not normalized:
  source: 0065 0301
     nfc: 00E9
localedata/locales/ig_NG:151: string not normalized:
  source: 0049 0301
     nfc: 00CD
localedata/locales/ig_NG:152: string not normalized:
  source: 0069 0301
     nfc: 00ED
localedata/locales/ig_NG:154: string not normalized:
  source: 004F 0301
     nfc: 00D3
localedata/locales/ig_NG:155: string not normalized:
  source: 006F 0301
     nfc: 00F3
localedata/locales/ig_NG:157: string not normalized:
  source: 0055 0301
     nfc: 00DA
localedata/locales/ig_NG:158: string not normalized:
  source: 0075 0301
     nfc: 00FA
localedata/locales/ig_NG:160: string not normalized:
  source: 0041 0300
     nfc: 00C0
localedata/locales/ig_NG:161: string not normalized:
  source: 0061 0300
     nfc: 00E0
localedata/locales/ig_NG:163: string not normalized:
  source: 0045 0300
     nfc: 00C8
localedata/locales/ig_NG:164: string not normalized:
  source: 0065 0300
     nfc: 00E8
localedata/locales/ig_NG:166: string not normalized:
  source: 0049 0300
     nfc: 00CC
localedata/locales/ig_NG:167: string not normalized:
  source: 0069 0300
     nfc: 00EC
localedata/locales/ig_NG:169: string not normalized:
  source: 004F 0300
     nfc: 00D2
localedata/locales/ig_NG:170: string not normalized:
  source: 006F 0300
     nfc: 00F2
localedata/locales/ig_NG:172: string not normalized:
  source: 0055 0300
     nfc: 00D9
localedata/locales/ig_NG:173: string not normalized:
  source: 0075 0300
     nfc: 00F9
localedata/locales/ik_CA:78: string not normalized:
  source: 004C 0323
     nfc: 1E36
localedata/locales/ik_CA:79: string not normalized:
  source: 006C 0323
     nfc: 1E37
localedata/locales/iso14651_t1_common:840: string not normalized:
  source: 0CBF 0CD5
     nfc: 0CC0
localedata/locales/iso14651_t1_common:841: string not normalized:
  source: 0CC6 0CD5
     nfc: 0CC7
localedata/locales/iso14651_t1_common:842: string not normalized:
  source: 0CC6 0CC2 0CD5
     nfc: 0CCB
localedata/locales/iso14651_t1_common:843: string not normalized:
  source: 0CCA 0CD5
     nfc: 0CCB
localedata/locales/iso14651_t1_common:844: string not normalized:
  source: 0CC6 0CC2
     nfc: 0CCA
localedata/locales/iso14651_t1_common:845: string not normalized:
  source: 0CC6 0CD6
     nfc: 0CC8
localedata/locales/iso14651_t1_common:928: string not normalized:
  source: 0BC6 0BBE
     nfc: 0BCA
localedata/locales/iso14651_t1_common:929: string not normalized:
  source: 0BC7 0BBE
     nfc: 0BCB
localedata/locales/iso14651_t1_common:930: string not normalized:
  source: 0BC6 0BD7
     nfc: 0BCC
localedata/locales/iso14651_t1_common:1052: string not normalized:
  source: 0D46 0D3E
     nfc: 0D4A
localedata/locales/iso14651_t1_common:1053: string not normalized:
  source: 0D47 0D3E
     nfc: 0D4B
localedata/locales/iso14651_t1_common:1054: string not normalized:
  source: 0D46 0D57
     nfc: 0D4C
localedata/locales/iso14651_t1_common:2106: string not normalized:
  source: 102D 1019 103A 1037
     nfc: 102D 1019 1037 103A
localedata/locales/iso14651_t1_common:2115: string not normalized:
  source: 102F 1019 103A 1037
     nfc: 102F 1019 1037 103A
localedata/locales/iso14651_t1_common:2559: string not normalized:
  source: 1025 102E
     nfc: 1026
localedata/locales/kw_GB:32: inappropriate character 'í' in string
localedata/locales/kw_GB:32: inappropriate character 'á' in string
localedata/locales/lb_LU:51: string not normalized:
  source: 0041 0308
     nfc: 00C4
localedata/locales/lb_LU:53: string not normalized:
  source: 004F 0308
     nfc: 00D6
localedata/locales/lb_LU:55: string not normalized:
  source: 0055 0308
     nfc: 00DC
localedata/locales/lb_LU:57: string not normalized:
  source: 0045 0308
     nfc: 00CB
localedata/locales/lb_LU:59: string not normalized:
  source: 0061 0308
     nfc: 00E4
localedata/locales/lb_LU:61: string not normalized:
  source: 006F 0308
     nfc: 00F6
localedata/locales/lb_LU:63: string not normalized:
  source: 0075 0308
     nfc: 00FC
localedata/locales/lb_LU:65: string not normalized:
  source: 0065 0308
     nfc: 00EB
localedata/locales/lg_UG:29: inappropriate escape sequence '/o'
localedata/locales/ln_CD:15: inappropriate character 'é' in string
localedata/locales/ln_CD:15: inappropriate character 'é' in string
localedata/locales/mai_IN:19: inappropriate escape sequence '/302'
localedata/locales/mg_MG:25: inappropriate escape sequence '/L'
localedata/locales/mni_IN:67: string not normalized:
  source: 09DF 09C1 09AE
     nfc: 09AF 09BC 09C1 09AE
localedata/locales/mni_IN:76: string not normalized:
  source: 09DF 09C1 09AE 09B6 0995 09C8 09B6 09BE
     nfc: 09AF 09BC 09C1 09AE 09B6 0995 09C8 09B6 09BE
localedata/locales/nb_NO:34: inappropriate character 'å' in string
localedata/locales/nb_NO:137: string not normalized:
  source: 0041 0308
     nfc: 00C4
localedata/locales/nb_NO:139: string not normalized:
  source: 0041 030A
     nfc: 00C5
localedata/locales/nb_NO:143: string not normalized:
  source: 004F 0308
     nfc: 00D6
localedata/locales/nb_NO:147: string not normalized:
  source: 0061 0308
     nfc: 00E4
localedata/locales/nb_NO:149: string not normalized:
  source: 0061 030A
     nfc: 00E5
localedata/locales/nb_NO:153: string not normalized:
  source: 006F 0308
     nfc: 00F6
localedata/locales/pa_IN:111: string not normalized:
  source: 0A36 0A41 0A71 0A15 0A30
     nfc: 0A38 0A3C 0A41 0A71 0A15 0A30
localedata/locales/pa_IN:112: string not normalized:
  source: 0A36 0A28 0A3F 0A71 0A1A 0A30
     nfc: 0A38 0A3C 0A28 0A3F 0A71 0A1A 0A30
localedata/locales/pa_IN:120: string not normalized:
  source: 0A36 0A41 0A71 0A15 0A30 0A35 0A3E 0A30
     nfc: 0A38 0A3C 0A41 0A71 0A15 0A30 0A35 0A3E 0A30
localedata/locales/pa_IN:121: string not normalized:
  source: 0A36 0A28 0A3F 0A71 0A1A 0A30 0A35 0A3E 0A30
     nfc: 0A38 0A3C 0A28 0A3F 0A71 0A1A 0A30 0A35 0A3E 0A30
localedata/locales/pa_IN:126: string not normalized:
  source: 0A5E 0A30 0A35 0A30 0A40
     nfc: 0A2B 0A3C 0A30 0A35 0A30 0A40
localedata/locales/pa_IN:155: string not normalized:
  source: 0A36 0A3E 0A2E
     nfc: 0A38 0A3C 0A3E 0A2E
localedata/locales/pap_CW:19: inappropriate character 'ç' in string
localedata/locales/pap_CW:27: inappropriate character 'ç' in string
localedata/locales/pl_PL:30: inappropriate escape sequence '/r'
localedata/locales/pl_PL:30: inappropriate escape sequence '/b'
localedata/locales/quz_PE:31: inappropriate escape sequence '/ '
localedata/locales/se_NO:25: inappropriate escape sequence '/a'
localedata/locales/se_NO:25: inappropriate escape sequence '/t'
localedata/locales/se_NO:25: inappropriate escape sequence '/l'
localedata/locales/se_NO:27: inappropriate character 'ø' in string
localedata/locales/sgs_LT:35: inappropriate character 'č' in string
localedata/locales/sgs_LT:86: string not normalized:
  source: 0070 0065 0074 006E 0069 0304 010D 0117
     nfc: 0070 0065 0074 006E 012B 010D 0117
localedata/locales/sgs_LT:118: string not normalized:
  source: 0074 0227 0304 0070
     nfc: 0074 01E1 0070
localedata/locales/sgs_LT:119: string not normalized:
  source: 006E 0065 0304
     nfc: 006E 0113
localedata/locales/sl_SI:29: inappropriate escape sequence '/M'
localedata/locales/sm_WS:90: inappropriate character '\t' in string
localedata/locales/sm_WS:92: inappropriate character '\t' in string
localedata/locales/sv_SE:122: string not normalized:
  source: 0041 0308
     nfc: 00C4
localedata/locales/sv_SE:124: string not normalized:
  source: 0041 030A
     nfc: 00C5
localedata/locales/sv_SE:128: string not normalized:
  source: 004F 0308
     nfc: 00D6
localedata/locales/sv_SE:132: string not normalized:
  source: 0061 0308
     nfc: 00E4
localedata/locales/sv_SE:134: string not normalized:
  source: 0061 030A
     nfc: 00E5
localedata/locales/sv_SE:138: string not normalized:
  source: 006F 0308
     nfc: 00F6
localedata/locales/ta_LK:23: inappropriate escape sequence '/36'
localedata/locales/ta_LK:23: inappropriate escape sequence '/1'
localedata/locales/ta_LK:24: inappropriate character '\t' in string
localedata/locales/translit_circle:231: string not normalized:
  source: 0028 1100 1161 0029
     nfc: 0028 AC00 0029
localedata/locales/translit_circle:233: string not normalized:
  source: 0028 1102 1161 0029
     nfc: 0028 B098 0029
localedata/locales/translit_circle:235: string not normalized:
  source: 0028 1103 1161 0029
     nfc: 0028 B2E4 0029
localedata/locales/translit_circle:237: string not normalized:
  source: 0028 1105 1161 0029
     nfc: 0028 B77C 0029
localedata/locales/translit_circle:239: string not normalized:
  source: 0028 1106 1161 0029
     nfc: 0028 B9C8 0029
localedata/locales/translit_circle:241: string not normalized:
  source: 0028 1107 1161 0029
     nfc: 0028 BC14 0029
localedata/locales/translit_circle:243: string not normalized:
  source: 0028 1109 1161 0029
     nfc: 0028 C0AC 0029
localedata/locales/translit_circle:245: string not normalized:
  source: 0028 110B 1161 0029
     nfc: 0028 C544 0029
localedata/locales/translit_circle:247: string not normalized:
  source: 0028 110C 1161 0029
     nfc: 0028 C790 0029
localedata/locales/translit_circle:249: string not normalized:
  source: 0028 110E 1161 0029
     nfc: 0028 CC28 0029
localedata/locales/translit_circle:251: string not normalized:
  source: 0028 110F 1161 0029
     nfc: 0028 CE74 0029
localedata/locales/translit_circle:253: string not normalized:
  source: 0028 1110 1161 0029
     nfc: 0028 D0C0 0029
localedata/locales/translit_circle:255: string not normalized:
  source: 0028 1111 1161 0029
     nfc: 0028 D30C 0029
localedata/locales/translit_circle:257: string not normalized:
  source: 0028 1112 1161 0029
     nfc: 0028 D558 0029
localedata/locales/translit_circle:259: string not normalized:
  source: 0028 110E 1161 11B7 1100 1169 0029
     nfc: 0028 CC38 ACE0 0029
localedata/locales/translit_circle:261: string not normalized:
  source: 0028 110C 116E 110B 1174 0029
     nfc: 0028 C8FC C758 0029
localedata/locales/translit_circle:263: string not normalized:
  source: 0028 110B 116E 0029
     nfc: 0028 C6B0 0029
localedata/locales/translit_combining:1859: string not normalized:
  source: 0928 093C
     nfc: 0929
localedata/locales/translit_combining:1861: string not normalized:
  source: 0930 093C
     nfc: 0931
localedata/locales/translit_combining:1863: string not normalized:
  source: 0933 093C
     nfc: 0934
localedata/locales/translit_combining:1881: string not normalized:
  source: 09C7 09BE
     nfc: 09CB
localedata/locales/translit_combining:1883: string not normalized:
  source: 09C7 09D7
     nfc: 09CC
localedata/locales/translit_combining:1903: string not normalized:
  source: 0B47 0B56
     nfc: 0B48
localedata/locales/translit_combining:1905: string not normalized:
  source: 0B47 0B3E
     nfc: 0B4B
localedata/locales/translit_combining:1907: string not normalized:
  source: 0B47 0B57
     nfc: 0B4C
localedata/locales/translit_combining:1913: string not normalized:
  source: 0B92 0BD7
     nfc: 0B94
localedata/locales/translit_combining:1915: string not normalized:
  source: 0BC6 0BBE
     nfc: 0BCA
localedata/locales/translit_combining:1917: string not normalized:
  source: 0BC7 0BBE
     nfc: 0BCB
localedata/locales/translit_combining:1919: string not normalized:
  source: 0BC6 0BD7
     nfc: 0BCC
localedata/locales/translit_combining:1921: string not normalized:
  source: 0C46 0C56
     nfc: 0C48
localedata/locales/translit_combining:1923: string not normalized:
  source: 0CBF 0CD5
     nfc: 0CC0
localedata/locales/translit_combining:1925: string not normalized:
  source: 0CC6 0CD5
     nfc: 0CC7
localedata/locales/translit_combining:1927: string not normalized:
  source: 0CC6 0CD6
     nfc: 0CC8
localedata/locales/translit_combining:1929: string not normalized:
  source: 0CC6 0CC2
     nfc: 0CCA
localedata/locales/translit_combining:1931: string not normalized:
  source: 0CC6 0CC2 0CD5
     nfc: 0CCB
localedata/locales/translit_combining:1933: string not normalized:
  source: 0D46 0D3E
     nfc: 0D4A
localedata/locales/translit_combining:1935: string not normalized:
  source: 0D47 0D3E
     nfc: 0D4B
localedata/locales/translit_combining:1937: string not normalized:
  source: 0D46 0D57
     nfc: 0D4C
localedata/locales/translit_combining:1939: string not normalized:
  source: 0DD9 0DCA
     nfc: 0DDA
localedata/locales/translit_combining:1941: string not normalized:
  source: 0DD9 0DCF
     nfc: 0DDC
localedata/locales/translit_combining:1943: string not normalized:
  source: 0DD9 0DCF 0DCA
     nfc: 0DDD
localedata/locales/translit_combining:1945: string not normalized:
  source: 0DD9 0DDF
     nfc: 0DDE
localedata/locales/translit_combining:1981: string not normalized:
  source: 1025 102E
     nfc: 1026
localedata/locales/translit_compat:161: string not normalized:
  source: 0FB2 0F81
     nfc: 0FB2 0F71 0F80
localedata/locales/translit_compat:163: string not normalized:
  source: 0FB3 0F81
     nfc: 0FB3 0F71 0F80
localedata/locales/translit_compat:1435: string not normalized:
  source: 0028 1100 1161 0029
     nfc: 0028 AC00 0029
localedata/locales/translit_compat:1437: string not normalized:
  source: 0028 1102 1161 0029
     nfc: 0028 B098 0029
localedata/locales/translit_compat:1439: string not normalized:
  source: 0028 1103 1161 0029
     nfc: 0028 B2E4 0029
localedata/locales/translit_compat:1441: string not normalized:
  source: 0028 1105 1161 0029
     nfc: 0028 B77C 0029
localedata/locales/translit_compat:1443: string not normalized:
  source: 0028 1106 1161 0029
     nfc: 0028 B9C8 0029
localedata/locales/translit_compat:1445: string not normalized:
  source: 0028 1107 1161 0029
     nfc: 0028 BC14 0029
localedata/locales/translit_compat:1447: string not normalized:
  source: 0028 1109 1161 0029
     nfc: 0028 C0AC 0029
localedata/locales/translit_compat:1449: string not normalized:
  source: 0028 110B 1161 0029
     nfc: 0028 C544 0029
localedata/locales/translit_compat:1451: string not normalized:
  source: 0028 110C 1161 0029
     nfc: 0028 C790 0029
localedata/locales/translit_compat:1453: string not normalized:
  source: 0028 110E 1161 0029
     nfc: 0028 CC28 0029
localedata/locales/translit_compat:1455: string not normalized:
  source: 0028 110F 1161 0029
     nfc: 0028 CE74 0029
localedata/locales/translit_compat:1457: string not normalized:
  source: 0028 1110 1161 0029
     nfc: 0028 D0C0 0029
localedata/locales/translit_compat:1459: string not normalized:
  source: 0028 1111 1161 0029
     nfc: 0028 D30C 0029
localedata/locales/translit_compat:1461: string not normalized:
  source: 0028 1112 1161 0029
     nfc: 0028 D558 0029
localedata/locales/translit_compat:1463: string not normalized:
  source: 0028 110C 116E 0029
     nfc: 0028 C8FC 0029
localedata/locales/translit_compat:1465: string not normalized:
  source: 0028 110B 1169 110C 1165 11AB 0029
     nfc: 0028 C624 C804 0029
localedata/locales/translit_compat:1467: string not normalized:
  source: 0028 110B 1169 1112 116E 0029
     nfc: 0028 C624 D6C4 0029
localedata/locales/wa_BE:69: string not normalized:
  source: 0041 030A
     nfc: 00C5
localedata/locales/wa_BE:70: string not normalized:
  source: 0061 030A
     nfc: 00E5
localedata/locales/yo_NG:80: string not normalized:
  source: 0045 0323
     nfc: 1EB8
localedata/locales/yo_NG:81: string not normalized:
  source: 0065 0323
     nfc: 1EB9
localedata/locales/yo_NG:86: string not normalized:
  source: 0045 0323 0301
     nfc: 1EB8 0301
localedata/locales/yo_NG:87: string not normalized:
  source: 0065 0323 0301
     nfc: 1EB9 0301
localedata/locales/yo_NG:88: string not normalized:
  source: 0045 0323 0300
     nfc: 1EB8 0300
localedata/locales/yo_NG:89: string not normalized:
  source: 0065 0323 0300
     nfc: 1EB9 0300
localedata/locales/yo_NG:91: string not normalized:
  source: 004F 0323
     nfc: 1ECC
localedata/locales/yo_NG:92: string not normalized:
  source: 006F 0323
     nfc: 1ECD
localedata/locales/yo_NG:97: string not normalized:
  source: 004F 0323 0301
     nfc: 1ECC 0301
localedata/locales/yo_NG:98: string not normalized:
  source: 006F 0323 0301
     nfc: 1ECD 0301
localedata/locales/yo_NG:99: string not normalized:
  source: 004F 0323 0300
     nfc: 1ECC 0300
localedata/locales/yo_NG:100: string not normalized:
  source: 006F 0323 0300
     nfc: 1ECD 0300
localedata/locales/yo_NG:102: string not normalized:
  source: 0053 0323
     nfc: 1E62
localedata/locales/yo_NG:103: string not normalized:
  source: 0073 0323
     nfc: 1E63
localedata/locales/yo_NG:105: string not normalized:
  source: 0041 0301
     nfc: 00C1
localedata/locales/yo_NG:106: string not normalized:
  source: 0061 0301
     nfc: 00E1
localedata/locales/yo_NG:107: string not normalized:
  source: 0045 0301
     nfc: 00C9
localedata/locales/yo_NG:108: string not normalized:
  source: 0065 0301
     nfc: 00E9
localedata/locales/yo_NG:109: string not normalized:
  source: 0049 0301
     nfc: 00CD
localedata/locales/yo_NG:110: string not normalized:
  source: 0069 0301
     nfc: 00ED
localedata/locales/yo_NG:111: string not normalized:
  source: 004D 0301
     nfc: 1E3E
localedata/locales/yo_NG:112: string not normalized:
  source: 006D 0301
     nfc: 1E3F
localedata/locales/yo_NG:113: string not normalized:
  source: 004E 0301
     nfc: 0143
localedata/locales/yo_NG:114: string not normalized:
  source: 006E 0301
     nfc: 0144
localedata/locales/yo_NG:115: string not normalized:
  source: 004F 0301
     nfc: 00D3
localedata/locales/yo_NG:116: string not normalized:
  source: 006F 0301
     nfc: 00F3
localedata/locales/yo_NG:117: string not normalized:
  source: 0055 0301
     nfc: 00DA
localedata/locales/yo_NG:118: string not normalized:
  source: 0075 0301
     nfc: 00FA
localedata/locales/yo_NG:120: string not normalized:
  source: 0041 0300
     nfc: 00C0
localedata/locales/yo_NG:121: string not normalized:
  source: 0061 0300
     nfc: 00E0
localedata/locales/yo_NG:122: string not normalized:
  source: 0045 0300
     nfc: 00C8
localedata/locales/yo_NG:123: string not normalized:
  source: 0065 0300
     nfc: 00E8
localedata/locales/yo_NG:124: string not normalized:
  source: 0049 0300
     nfc: 00CC
localedata/locales/yo_NG:125: string not normalized:
  source: 0069 0300
     nfc: 00EC
localedata/locales/yo_NG:128: string not normalized:
  source: 004E 0300
     nfc: 01F8
localedata/locales/yo_NG:129: string not normalized:
  source: 006E 0300
     nfc: 01F9
localedata/locales/yo_NG:130: string not normalized:
  source: 004F 0300
     nfc: 00D2
localedata/locales/yo_NG:131: string not normalized:
  source: 006F 0300
     nfc: 00F2
localedata/locales/yo_NG:132: string not normalized:
  source: 0055 0300
     nfc: 00D9
localedata/locales/yo_NG:133: string not normalized:
  source: 0075 0300
     nfc: 00F9
localedata/locales/yo_NG:140: string not normalized:
  source: 0041 0302
     nfc: 00C2
localedata/locales/yo_NG:141: string not normalized:
  source: 0061 0302
     nfc: 00E2
localedata/locales/yo_NG:142: string not normalized:
  source: 0045 0302
     nfc: 00CA
localedata/locales/yo_NG:143: string not normalized:
  source: 0065 0302
     nfc: 00EA
localedata/locales/yo_NG:144: string not normalized:
  source: 1EB8 0302
     nfc: 1EC6
localedata/locales/yo_NG:145: string not normalized:
  source: 1EB9 0302
     nfc: 1EC7
localedata/locales/yo_NG:146: string not normalized:
  source: 0049 0302
     nfc: 00CE
localedata/locales/yo_NG:147: string not normalized:
  source: 0069 0302
     nfc: 00EE
localedata/locales/yo_NG:148: string not normalized:
  source: 004F 0302
     nfc: 00D4
localedata/locales/yo_NG:149: string not normalized:
  source: 006F 0302
     nfc: 00F4
localedata/locales/yo_NG:150: string not normalized:
  source: 1ECC 0302
     nfc: 1ED8
localedata/locales/yo_NG:151: string not normalized:
  source: 1ECD 0302
     nfc: 1ED9
localedata/locales/yo_NG:152: string not normalized:
  source: 0055 0302
     nfc: 00DB
localedata/locales/yo_NG:153: string not normalized:
  source: 0075 0302
     nfc: 00FB
localedata/locales/yo_NG:155: string not normalized:
  source: 0041 030C
     nfc: 01CD
localedata/locales/yo_NG:156: string not normalized:
  source: 0061 030C
     nfc: 01CE
localedata/locales/yo_NG:157: string not normalized:
  source: 0045 030C
     nfc: 011A
localedata/locales/yo_NG:158: string not normalized:
  source: 0065 030C
     nfc: 011B
localedata/locales/yo_NG:161: string not normalized:
  source: 0049 030C
     nfc: 01CF
localedata/locales/yo_NG:162: string not normalized:
  source: 0069 030C
     nfc: 01D0
localedata/locales/yo_NG:163: string not normalized:
  source: 004F 030C
     nfc: 01D1
localedata/locales/yo_NG:164: string not normalized:
  source: 006F 030C
     nfc: 01D2
localedata/locales/yo_NG:167: string not normalized:
  source: 0055 030C
     nfc: 01D3
localedata/locales/yo_NG:168: string not normalized:
  source: 0075 030C
     nfc: 01D4
-------------- next part --------------
A non-text attachment was scrubbed...
Name: check-localedef.py
Type: text/x-python-script
Size: 15816 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20170726/452e2022/attachment.bin>

More information about the Libc-alpha mailing list