Unconsistent command-line parsing in case of UTF-8 quoted arguments

Andrey Repin anrdaemon@yandex.ru
Sun Oct 4 11:18:09 GMT 2020

Greetings, Jérôme Froissart!

> By discussing a merge request on another project [1], I think
> billziss-gh found a weirdness in the way Cygwin parses the command
> line arguments when non-ASCII characters come into play.

> cygwin should parse the following command line
>     binary.exe --non-ascii "charaçtérs" --ascii "nothing-fancy-here"
> as
>     argv = ["binary.exe",
>             "--non-ascii",
>             "chara\xXX\xXXt\xXX\xXXrs",
>             "--ascii",
>             "nothing-fancy-here"]
>     // \xXX\xXX being the UTF-8 encoding of the special characters,
> but this does not really matter here
> before calling main()

> it parses it as
>     argv = ["binary.exe",
>             "--non-ascii",
>             "\"chara\xXX\xXXt\xXX\xXXrs\"", // mind the unstripped
> quotes here...
>             "--ascii",
>             "nothing-fancy-here" // ...but not here
>     ]

> It looks that words containing UTF-8 characters are not properly
> stripped when they are surrounded by quotes, unlinke ASCII words.

> More examples and a better description is available at [1] (thanks to
> billziss-gh for his analysis, much more thorough than mine)
> For the record, we wrote a work-around in our specific program, but
> handling this issue in Cygwin might be a better way to solve it.

> [1]: https://github.com/billziss-gh/sshfs-win/pull/208 (Checking for
> quotes around non-ascii usernames passed by Windows)

> Thanks for your help! In case you didn't have time, please tell me
> where to look at, and I might try to fix it myself and send a patch
> proposal if that is easy enough (I have never read Cygwin's code yet).

This seems like the Cygwin command was launched from a non-Cygwin terminal or
from a terminal where locale was not set to UNICODE.

Please provide the results of "locale" command right before running your test

With best regards,
Andrey Repin
Sunday, October 4, 2020 14:16:17

Sorry for my terrible english...

More information about the Cygwin mailing list