[1.7] Problem with national characters in directory names when using UTF-8 charset

[Alexey Borzenkov]

There is something strange going on with national characters in
directory names when using Cygwin 1.7 with UTF-8. Here's a sample
session:

# test.rb
# -*- coding: utf-8 -*-
filename = File.expand_path("test.txt")
puts filename
puts File.open(filename) { |f| f.read }

# test.txt
This is a test

C:\cygwin\home\aborzenkov> set LANG=en_US.UTF-8

C:\cygwin\home\aborzenkov> mkdir проверка

C:\cygwin\home\aborzenkov> copy test.rb проверка

C:\cygwin\home\aborzenkov> copy test.txt проверка

C:\cygwin\home\aborzenkov> C:\cygwin\bin\ruby проверка/test.rb
/usr/bin/ruby: No such file or directory -- проверка/test.rb (LoadError)

C:\cygwin\home\aborzenkov> C:\cygwin\bin\cat проверка/test.txt
This is a test

C:\cygwin\home\aborzenkov> cd проверка

C:\cygwin\home\aborzenkov\проверка> C:\cygwin\bin\ruby test.rb
/home/aborzenkov/▒??▒N?▒??▒??▒?ч▒N?▒??▒?°/test.txt
This is a test

C:\cygwin\home\aborzenkov\проверка> C:\cygwin\bin\cat test.txt
/usr/bin/cat: test.txt: No such file or directory

C:\cygwin\home\aborzenkov\проверка> C:\cygwin\bin\ls -al
/usr/bin/ls: cannot open directory .: No such file or directory

Why is it that some commands can't accept russian character in
filenames, yet work within russian directories, and other can open
filenames with russian paths, but can't work within russian
directories? It seems extremely weird to me. :-/ Also, I'm wondering
about this discrepancy:

C:\cygwin\home\aborzenkov> C:\cygwin\bin\ruby /bin/irb
irb(main):001:0> Dir.chdir("проверка")
=> 0
irb(main):002:0> File.expand_path("*")
=> "/home/aborzenkov/\320\277\321\200\320\276\320\262\320\265\321\200\320\272\320\260/*"

C:\cygwin\home\aborzenkov\проверка> C:\cygwin\bin\ruby /bin/irb
irb(main):001:0> File.expand_path("*")
=> "/home/aborzenkov/\016\320\277\016\321\200\016\320\276\016\320\262\016\320\265\016\321\200\016\320\272\016\320\260/*"

Notice how for the same current directory (one where cygwin session
has done chdir to russian directory on its own, another where cygwin
session was started in russian directory) give different results for
File.expand_path in ruby. If I understood cygwin documentation
correctly, \016 is supposed to appear only for character that cannot
be represented with current charset (which is utf-8), yet in second
case they appear all over the place. The same thing is happening with,
for example, bash, which shows garbled pwd output when started from
within russian directory, yet works well when I chdir to that
directory manually.

What's going on?

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/