Re: support C/C++ identifiers named with non-ASCII characters

[Simon Marchi <simark at simark dot ca>]

On 2018-05-21 04:52 AM, 張俊芝 wrote:
> Hello, team.
> 
> This patch fixes the bug at
> https://sourceware.org/bugzilla/show_bug.cgi?id=22973 .
> 
> Here is how to test the patch:
> 
> Step 1. If you are using Clang or any other C compilers that have 
> implemented
>          support for Unicode identifiers, then create a C file with the 
> following
>          content:
> 
> int main(int 參量, char* 參[])
> {
>    struct 集
>    {
>      int 數[3];
>    } 集 = {100, 200, 300};
>    int 序 = 2;
>    return 0;
> }
> 
> Or if you are using GCC, create a C file with the following content as a
> workaround(GCC still doesn't actually support Unicode identifiers in 
> 2018, which
> is a pity):
> 
> int main(int \u53C3\u91CF, char* \u53C3[])
> {
>    struct \u96C6
>    {
>      int \u6578[3];
>    } \u96C6 = {100, 200, 300};
>    int \u5E8F = 2;
>    return 0;
> }
> 
> Step 2. Compile the C file.
> 
> Step 3. Run GDB for the compiled executable, add a breakpoint in "return 0".
> 
> Step 4. Run until the breakpoint.
> 
> Step 5. Test the following commands to see if they work:
>          p 參量
>          p 參
>          p 集
>          p 集.數
>          p 集.數[序]
> 
> Thanks for your review.
> 

Hi Zhang,

Thanks for the patch, I tested it quickly, it seems to work as expected.

Could you please write a small test case in testsuite/gdb.base with the example
you gave, so we make sure this doesn't get broken later?  If you can write it
in such a way that both clang and gcc understand it would be better, because
most people run the testuite using gcc to compile test programs.

I am not a specialist in lexing and parsing C, so can you explain quickly why
you think this is a good solution?  Quickly, I understand that you change the
identifier recognition algorithm to a blacklist of characters rather than
a whitelist, so bytes that are not recognized (such as those that compose
the utf-8 encoded characters) are not rejected.

Given unlimited time, would the right solution be to use a lib to parse the
string as utf-8, and reject strings that are not valid utf-8?

Here are some not and formatting comments:

> +static bool is_identifier_separator (char);

You don't have to forward declare the function if it's not necessary.

> +    /* '<' should not be a token separator, because it can be an open angle
> +       bracket followed by a nested template identifier in C++. */

Please use two spaces after the final period (...C++.  */).

> +  if (is_identifier_separator(c))

Please use a space before the parentheses:

  is_identifier_separator (c)


> +  for (c = tokstart[namelen]; !is_identifier_separator(c);)

Here too.

Thanks!

Simon