[Bug regex/23393] Handle [a-z] and [A-Z] in consistent portable fashion regardless of locale.

["cvs-commit at gcc dot gnu.org" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=23393

--- Comment #34 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU C Library master sources".

The branch, master has been updated
       via  7cd7d36f1feb3ccacf476e909b115b45cdd46e77 (commit)
      from  3fb455b318c48f76280b4f8a0ff23b2cb1af9a3e (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=7cd7d36f1feb3ccacf476e909b115b45cdd46e77

commit 7cd7d36f1feb3ccacf476e909b115b45cdd46e77
Author: Carlos O'Donell <carlos@redhat.com>
Date:   Wed Jul 25 17:00:45 2018 -0400

    Keep expected behaviour for [a-z] and [A-z] (Bug 23393).

    In commit 9479b6d5e08eacce06c6ab60abc9b2f4eb8b71e4 we updated all of
    the collation data to harmonize with the new version of ISO 14651
    which is derived from Unicode 9.0.0.  This collation update brought
    with it some changes to locales which were not desirable by some
    users, in particular it altered the meaning of the
    locale-dependent-range regular expression, namely [a-z] and [A-Z], and
    for en_US it caused uppercase letters to be matched by [a-z] for the
    first time.  The matching of uppercase letters by [a-z] is something
    which is already known to users of other locales which have this
    property, but this change could cause significant problems to en_US
    and other similar locales that had never had this change before.
    Whether this behaviour is desirable or not is contentious and GNU Awk
    has this to say on the topic:
    https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html
    While the POSIX standard also has this further to say: "RE Bracket
    Expression":
    http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap09.html
    "The current standard leaves unspecified the behavior of a range
    expression outside the POSIX locale. ... As noted above, efforts were
    made to resolve the differences, but no solution has been found that
    would be specific enough to allow for portable software while not
    invalidating existing implementations."
    In glibc we implement the requirement of ISO POSIX-2:1993 and use
    collation element order (CEO) to construct the range expression, the
    API internally is __collseq_table_lookup().  The fact that we use CEO
    and also have 4-level weights on each collation rule means that we can
    in practice reorder the collation rules in iso14651_t1_common (the new
    data) to provide consistent range expression resolution *and* the
    weights should maintain the expected total order.  Therefore this
    patch does three things:

    * Reorder the collation rules for the LATIN script in
      iso14651_t1_common to deinterlace uppercase and lowercase letters in
      the collation element orders.

    * Adds new test data en_US.UTF-8.in for sort-test.sh which exercises
      strcoll* and strxfrm* and ensures the ISO 14651 collation remains.

    * Add back tests to tst-fnmatch.input and tst-regexloc.c which
      exercise that [a-z] does not match A or Z.

    The reordering of the ISO 14651 data is done in an entirely mechanical
    fashion using the following program attached to the bug:
    https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c28

    It is up for discussion if the iso14651_t1_common data should be
    refined further to have 3 very tight collation element ranges that
    include only a-z, A-Z, and 0-9, which would implement the solution
    sought after in:
    https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c12
    and implemented here:
    https://www.sourceware.org/ml/libc-alpha/2018-07/msg00854.html

    No regressions on x86_64.
    Verified that removal of the iso14651_t1_common change causes tst-fnmatch
    to regress with:
    422: fnmatch ("[a-z]", "A", 0) = 0 (FAIL, expected FNM_NOMATCH) ***
    ...
    425: fnmatch ("[A-Z]", "z", 0) = 0 (FAIL, expected FNM_NOMATCH) ***

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog                             |   11 +
 localedata/Makefile                   |    1 +
 localedata/locales/iso14651_t1_common | 1928 ++++++++++++++++----------------
 posix/tst-fnmatch.input               |  125 ++-
 posix/tst-regexloc.c                  |    8 +-
 5 files changed, 1065 insertions(+), 1008 deletions(-)

-- 
You are receiving this mail because:
You are on the CC list for the bug.