Bug 11532 - Support old DOS Lithuanian character sets in iconv
Summary: Support old DOS Lithuanian character sets in iconv
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: localedata (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: GNU C Library Locale Maintainers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-04-23 14:24 UTC by Rimas Kudelis
Modified: 2014-06-30 09:20 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments
Mapping of CP770 (999 bytes, text/plain)
2010-05-04 09:12 UTC, Rimas Kudelis
Details
Mapping of CP771 (960 bytes, text/plain)
2010-05-04 09:13 UTC, Rimas Kudelis
Details
Mapping of CP772 (969 bytes, text/plain)
2010-05-04 09:13 UTC, Rimas Kudelis
Details
Mapping of CP773 (977 bytes, text/plain)
2010-05-04 09:13 UTC, Rimas Kudelis
Details
Mapping of CP774 (988 bytes, text/plain)
2010-05-04 09:14 UTC, Rimas Kudelis
Details
Mapping of CP773 (corrected) (977 bytes, text/plain)
2010-05-04 11:08 UTC, Rimas Kudelis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rimas Kudelis 2010-04-23 14:24:55 UTC
Back in DOS days, a few different character sets were more or less used in
Lithuania: 770, 771, 772, 773, 774, and 775. Iconv currently only supports the
latter. It would be nice to get support for others on the list too.

Links:
http://www.likit.lt/nostyle/770.htm
http://www.likit.lt/nostyle/771.htm
http://www.likit.lt/nostyle/772.htm
<no illustration for cp773>
http://www.likit.lt/nostyle/774.htm

If adding these character sets to iconv is generally acceptable, I think I could
try to generate mappings from all these charsets to UTF-8
Comment 1 Ulrich Drepper 2010-05-03 17:01:50 UTC
Then provide mapping tables.
Comment 2 Rimas Kudelis 2010-05-04 09:12:57 UTC
Created attachment 4762 [details]
Mapping of CP770
Comment 3 Rimas Kudelis 2010-05-04 09:13:16 UTC
Created attachment 4763 [details]
Mapping of CP771
Comment 4 Rimas Kudelis 2010-05-04 09:13:33 UTC
Created attachment 4764 [details]
Mapping of CP772
Comment 5 Rimas Kudelis 2010-05-04 09:13:49 UTC
Created attachment 4765 [details]
Mapping of CP773
Comment 6 Rimas Kudelis 2010-05-04 09:14:08 UTC
Created attachment 4766 [details]
Mapping of CP774
Comment 7 Rimas Kudelis 2010-05-04 09:28:45 UTC
I've attached five files with mapping tables for each codepage.
Their format is: 
[octal code]: [UTF-8 character]

Lower 127 positions (0000-0177) match ASCII in all cases, so only the positions
starting 0200 matter.

It seems like these charsets are (or maybe were) supported by ICU (see [1]). The
page also has some further descriptions that could be used when forming alias
names for cp77x charsets:

CP770 	Lithuanian Standard RST 1095-89
CP771 	KBL (Lithuanian and Russian characters)
CP772 	Lithuanian Standard LST 1284:1993
CP773 	Lithuanian (Mix of 771 and 775)
CP774 	Lithuanian Standard 1283:1993

Unfortunately, I couldn't find source files of ICU mappings of these character
sets at [2], so I can't attach them. Instead, I used a small program found at
[3], developed a few years ago specifically to act as a converter among
different character sets used in Lithuania (note: I changed one symbol in
CP770.txt to match with the actual standard).

If it's possible to find ICU mappings, I think most likely they should be used
as a basis for conversion. Otherwise, the files attached should be fine.

[1]
http://publib.boulder.ibm.com/infocenter/tivihelp/v24r1/index.jsp?topic=/com.ibm.itcama.doc_6.2.3/itcam_oraclerac63200.htm
[2] http://source.icu-project.org/repos/icu/data/trunk/charset/data/
[3] https://www3.mruni.lt/~rims/kodav/#Diegimas
Comment 8 Rimas Kudelis 2010-05-04 11:08:37 UTC
Created attachment 4767 [details]
Mapping of CP773 (corrected)

According to the name mentioned on ICU page, and cp773.acm (CP773 mapping for
Linux console) found on [1], this is the more correct mapping of codepage 773.

[1] http://gedmin.as/lit-con/
Comment 9 Ulrich Drepper 2011-05-10 03:16:46 UTC
The files aren't usable in that form.  It was quite a lot of work to make all the transformations.  Support is in git now.
Comment 10 Rimas Kudelis 2011-05-10 05:05:18 UTC
Wow, thanks!

I've actually got hold of the paper standards, at least some of them, so I should be able to check the validity of our mappings when time permits.