[newlib-cygwin/cygwin-3_6-branch] Cygwin: is_console_app(): deal with the `.bat`/`.cmd` file extensions first

[Takashi Yano]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=a83652b67c1422ae3bf2405b429603fdb87a5202

commit a83652b67c1422ae3bf2405b429603fdb87a5202
Author: Johannes Schindelin <johannes.schindelin@gmx.de>
Date:   Mon Dec 15 17:06:34 2025 +0000

    Cygwin: is_console_app(): deal with the `.bat`/`.cmd` file extensions first
    
    This function contains special handling of these file extensions,
    treating them as console applications always, even if the first 1024
    bytes do not contain a PE header with the console bits set.
    
    However, Batch and Command files are never expected to have such a
    header, therefore opening them and reading their first bytes is a waste
    of time.
    
    Let's honor the best practice to deal with easy conditions that allow
    early returns first.
    
    Fixes: bb4285206207 (Cygwin: pty: Implement new pseudo console suppot., 2020-08-19)
    Reviewed-by: Takashi Yano <takashi.yano@nifty.ne.jp>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    (cherry picked from commit 18b3c0fd0a26b3fc1da720e17103512b90eb71fc)

Diff:
---
 winsup/cygwin/fhandler/termios.cc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/winsup/cygwin/fhandler/termios.cc b/winsup/cygwin/fhandler/termios.cc
index 61d01e931..8f98e42a4 100644
--- a/winsup/cygwin/fhandler/termios.cc
+++ b/winsup/cygwin/fhandler/termios.cc
@@ -704,6 +704,9 @@ fhandler_termios::fstat (struct stat *buf)
 static bool
 is_console_app (const WCHAR *filename)
 {
+  wchar_t *e = wcsrchr (filename, L'.');
+  if (e && (wcscasecmp (e, L".bat") == 0 || wcscasecmp (e, L".cmd") == 0))
+    return true;
   HANDLE h;
   h = CreateFileW (filename, GENERIC_READ, FILE_SHARE_READ,
 		   NULL, OPEN_EXISTING, 0, NULL);
@@ -720,9 +723,6 @@ is_console_app (const WCHAR *filename)
   IMAGE_NT_HEADERS32 *p = (IMAGE_NT_HEADERS32 *) memmem (buf, n, "PE\0\0", 4);
   if (p && (char *) &p->OptionalHeader.DllCharacteristics <= buf + n)
     return p->OptionalHeader.Subsystem == IMAGE_SUBSYSTEM_WINDOWS_CUI;
-  wchar_t *e = wcsrchr (filename, L'.');
-  if (e && (wcscasecmp (e, L".bat") == 0 || wcscasecmp (e, L".cmd") == 0))
-    return true;
   /* Return true for unknown to avoid standard handles from being unset.
      Setting-up standard handles for GUI apps is pointless, but not unsafe. */
   return true;