shell expansion produces e.g. "ls: cannot access '*.pdf': No such file or directory" in Windows CMD shell, but works okay in bash

[Paul Moore]

Interesting. Maybe codepage-related issues, then. Sorry, I'm out of my
depth now, I'll leave it to someone else to diagnose further.

On Sun, 22 Mar 2020 at 19:54, Jay Libove <libove@felines.org> wrote:
>
> Good suggestion, deleting files one by one. It's not just one file, but it does seem to have something to do with some file name patterns.
> I think I've got it. It's accented characters.
> I live in Spain. Spanish has accented characters such as "Asociación".
> When I remove all files containing any accented character in their name, the problem goes away.
> So the theory now is that the Cygwin argv-processing code has a problem with áccented charàcters ...
> -Jay
>
> -----Original Message-----
> From: Paul Moore <p.f.moore@gmail.com>
> Sent: Sunday, 22 March 2020 20:42
> To: Jay Libove <libove@felines.org>
> Cc: cygwin@cygwin.com
> Subject: Re: shell expansion produces e.g. "ls: cannot access '*.pdf': No such file or directory" in Windows CMD shell, but works okay in bash
>
> Have you tried deleting files one by one, to see if the issue is related to a single file (sorry if this is an obvious suggestion that you've already tried).
>
> In Cygwin bash, it's the shell that glob-expands wildcards before calling your program (e.g. ls), and in find, it's the find code that does the glob matching. But when running Cygwin utilities from a Windows shell, it's the Cygwin argv-processing code linked into the executable that does the glob-expansion. So it's reasonable to me that you should see the issue only with CMD, and not with bash or find. But that only confirms what bit of code is involved - not what the actual problem is :-(
>
> I don't actually know much about how the cygwin glob code actually works (my main involvement with it has been to confirm that it doesn't suit my specific needs, and to work out a way to bypass it...) so I can only offer fairly basic suggestions, I'm afraid...
>
> Paul
>
> On Sun, 22 Mar 2020 at 19:27, Jay Libove <libove@felines.org> wrote:
> >
> > Thanks Paul, both for your initial reply, and your follow-up.
> >
> > In this case it's not a matter case sensitivity.
> > I've verified that, in one of the example cases, there are both *.pdf and *.PDF files in the subject directory.
> > Both 'ls *.pdf' and 'ls *.PDF' produce the "ls: cannot access '*.whatever': No such file or directory" error.
> >
> > (Nor, to the other respondent's question, as I pointed out in my original post, is it ACLs, as I did check CACLS before posting).
> >
> > I also tried copying (using Windows CMD "COPY") *.pdf (so being under Windows, not Cygwin, it matches all cases) from a subject directory to a new test directory.
> > In the resulting copy in the new test directory, the Cygwin shell expansion problem persists.
> >
> > Here's an interesting twist:
> > C:> cd c:\bin\cygwin64\bin
> > C:> ln gnufind.exe find.exe # I do this to allow me to differentiate between Windows' built-in very limited FIND command, and GNU/Cygwin's far superior find command.
> > C:> cd \my\test\directory
> > C:> gnufind . -name *.pdf -print
> > [ successfully returns all *.pdf {lower case only} files in the
> > subject directory ] C:> gnufind . -name *.PDF -print [ successfully
> > returns all *.pdf {upper case only} files in the subject directory ]
> >
> > I'm pretty sure that Cygwin 'find' does NOT try to emulate shell globbing the way 'ls' does, so it makes sense that this works, and it supports the theory that something weird is going on between how Cygwin does shell expansion when under Windows CMD vs. when fully within the Cygwin environment (under bash where of course bash is doing the shell expansion, and ls or other Cygwin commands don't have to).
> >
> > Does any of this help pinpoint the problem further?
> >
> > thanks again,
> > -Jay
> >
> > -----Original Message-----
> > From: Paul Moore <p.f.moore@gmail.com>
> > Sent: Sunday, 22 March 2020 20:09
> > To: Jay Libove <libove@felines.org>
> > Cc: cygwin@cygwin.com
> > Subject: Re: shell expansion produces e.g. "ls: cannot access '*.pdf':
> > No such file or directory" in Windows CMD shell, but works okay in
> > bash
> >
> > Is this because cygwin globbing is (by default) case sensitive? You could set the CYGWIN environment variable to "glob:ignorecase" to get case-insensitive behaviour.
> >
> > Paul
> >
> > On Sun, 22 Mar 2020 at 17:52, Jay Libove via Cygwin <cygwin@cygwin.com> wrote:
> > >
> > > I've never seen this before.
> > > In a Windows CMD shell, Cygwin shell expansion, for example:
> > > ls *.pdf
> > >
> > > returns:
> > > ls: cannot access '*.PDF': No such file or directory (Indeed, any
> > > Cygwin shell expansion, when executed from within Windows CMD,
> > > produces this error. See below)
> > >
> > > ls *someotherwildcard* (that matches the same .pdf files) DOES return the expected file list.
> > >
> > > Example:
> > >
> > > C:> DIR *.pdf
> > > Volume in drive C is C
> > > Volume Serial Number is 8674-712A
> > >
> > > Directory of C:\Temp
> > >
> > > 22/03/2020  18:30         1.675.954 test.pdf
> > > XX/XX/XXXX  XX:XX         {Any many other .pdf files}
> > >
> > > Yet:
> > >
> > > C:> ls *.pdf
> > > ls: cannot access '*.pdf': No such file or directory
> > >
> > > And:
> > > C:> bash
> > > user@hostname /cygdrive/C/Temp/test
> > > $ ls *.pdf
> > > A.pdf
> > > B.pdf
> > > {etc}
> > >
> > > And, not ALL of the *.pdf files in the particular directory where I've encountered this trigger the problem...
> > >
> > > C:> ls N*.pdf
> > > N.pdf
> > >
> > > C:> ls A*.pdf
> > > ls: cannot access 'A*.pdf': No such file or directory
> > >
> > > Nor do all directories containing .pdf files produce this. Of the many thousands of files and directories that I have, only some produce this problem.
> > > In others, ls *.pdf works perfectly in Windows CMD.
> > >
> > > I've looked at the Windows ATTRIB and CACLS of the files in directories where this problem occurs.
> > > They're all the same. That is, uniform across all files and directories. Nothing interesting.
> > >
> > > It's not just 'ls':
> > >
> > > C:> cat *.pdf
> > > cat: '*.pdf': No such file or directory
> > >
> > > So, it appears to be Cygwin shell expansion, when executed under Windows CMD, which is provoking this strange behavior.
> > > Any ideas what could be causing this, and how to solve it?
> > >
> > > many thanks,
> > > Jay
> > >
> > > --
> > > Problem reports:      https://cygwin.com/problems.html
> > > FAQ:                  https://cygwin.com/faq/
> > > Documentation:        https://cygwin.com/docs.html
> > > Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple