[newlib-cygwin/cygwin-3_5-branch] Cygwin: try_to_bin: transpose deleted file name to valid Unicode chars

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=bb854e0f112ce69c52e0511bbfc423577c57f2a0

commit bb854e0f112ce69c52e0511bbfc423577c57f2a0
Author:     Corinna Vinschen <corinna@vinschen.de>
AuthorDate: Wed Dec 4 22:54:18 2024 +0100
Commit:     Corinna Vinschen <corinna@vinschen.de>
CommitDate: Wed Dec 4 22:56:37 2024 +0100

    Cygwin: try_to_bin: transpose deleted file name to valid Unicode chars
    
    Since commit 314c2d2fedc5f ("* syscalls.cc (try_to_bin): Handle remote
    shares as well.") try_to_bin() transposes the .cyg prefix for temporary
    files to invalid low surrogate halfs on filesystems setting the
    FILE_UNICODE_ON_DISK flag.
    
    This works on NTFS, but not necessarily on other filesystems, which often
    require all chars in a filename to be valid Unicode chars.  Fix this by
    transposing into the private use area instead.
    
    Fixes: 314c2d2fedc5f ("* syscalls.cc (try_to_bin): Handle remote shares as well.")
    Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
    (cherry picked from commit 0924d5f1078b368b6e290d11c9f3d57bc7767576)

Diff:
---
 winsup/cygwin/release/3.5.5 |  3 +++
 winsup/cygwin/syscalls.cc   | 10 +++++-----
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/winsup/cygwin/release/3.5.5 b/winsup/cygwin/release/3.5.5
index 7ccf28abf317..d5063af1640d 100644
--- a/winsup/cygwin/release/3.5.5
+++ b/winsup/cygwin/release/3.5.5
@@ -51,3 +51,6 @@ Fixes:
 
 - Fix frequent page fault caused in Windows Terminal.
   Addresses: https://cygwin.com/pipermail/cygwin/2024-December/256841.html
+
+- Fix using invalid chars in temporary file names for deleted files.
+  Addresses: https://cygwin.com/pipermail/cygwin/2024-November/256813.html
diff --git a/winsup/cygwin/syscalls.cc b/winsup/cygwin/syscalls.cc
index 1d9308ef46cb..11033bca5e6b 100644
--- a/winsup/cygwin/syscalls.cc
+++ b/winsup/cygwin/syscalls.cc
@@ -340,14 +340,14 @@ try_to_bin (path_conv &pc, HANDLE &fh, ACCESS_MASK access, ULONG flags)
   else
     {
       /* Create unique filename.  Start with a dot, followed by "cyg"
-	 transposed into the Unicode low surrogate area (U+dc00) on file
-	 systems supporting Unicode (except Samba), followed by the inode
-	 number in hex, followed by a path hash in hex.  The combination
-	 allows to remove multiple hardlinks to the same file. */
+	 transposed to the Unicode private use area in the U+f700 area
+	 on file systems supporting Unicode (except Samba), followed by
+	 the inode number in hex, followed by a path hash in hex.  The
+	 combination allows to remove multiple hardlinks to the same file. */
       RtlAppendUnicodeToString (&recycler,
 				(pc.fs_flags () & FILE_UNICODE_ON_DISK
 				 && !pc.fs_is_samba ())
-				? L".\xdc63\xdc79\xdc67" : L".cyg");
+				? L".\xf763\xf779\xf767" : L".cyg");
       pfii = (PFILE_INTERNAL_INFORMATION) infobuf;
       status = NtQueryInformationFile (fh, &io, pfii, sizeof *pfii,
 				       FileInternalInformation);