Bug 12378

POSIX 2008 states for fnmatch:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html

"The fnmatch() function shall match patterns as described in XCU Patterns Matching a Single Character and Patterns Matching Multiple Characters."

which in turn states:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13_01

"If an open bracket introduces a bracket expression as in XBD RE Bracket Expression , except that the <exclamation-mark> character ( '!' ) shall replace the <circumflex> character ( '^' ) in its role in a non-matching list in the regular expression notation, it shall introduce a pattern bracket expression. A bracket expression starting with an unquoted <circumflex> character produces unspecified results. Otherwise, '[' shall match the character itself."

This wording was intentionally changed over the older POSIX 2001 wording ("The open bracket shall introduce a pattern bracket expression.") -- in particular via the addition of the last sentence about '[' matching itself -- because of the fact that the standard requires the existence of '[' as a utility available via PATH searches.  After all, simple shell commands such as '[ -n "$1" ]' do not invoke file-name pattern expansions with undefined behavior due to a non-terminated bracket expression, but instead invoke the '[' utility.

Accordingly, when the pattern given to fnmatch consists of a '[' that does not introduce a bracket expression, then that pattern shall match a literal '[' without requiring escaping.  Thus, fnmatch("[", "[", 0) is required to return 0 (and in fact, it does so on at least Cygwin (BSD code base) and Solaris 10).  But on glibc 12.90, it mistakenly returns FNM_NOMATCH (which is itself a bit surprising - it seems like fnmatch() should return something _other_ than FNM_NOMATCH if it thinks an input pattern is invalid), such as how Solaris has FNM_ERROR==2).

$ cat > foo.c <<\EOF
#include <fnmatch.h>
int main (void) { return fnmatch ("[", "[", 0); }
EOF
$ gcc -o foo foo.c
$ ./foo
$ echo $? # should be 0
1