Bug 23926 - fnmatch with FNM_PATHNAME is not POSIX compliant
Summary: fnmatch with FNM_PATHNAME is not POSIX compliant
Status: RESOLVED INVALID
Alias: None
Product: glibc
Classification: Unclassified
Component: glob (show other bugs)
Version: 2.28
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-11-27 00:02 UTC by Cyker Way
Modified: 2018-11-30 18:10 UTC (History)
0 users

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments
test case (181 bytes, text/x-csrc)
2018-11-27 00:02 UTC, Cyker Way
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Cyker Way 2018-11-27 00:02:37 UTC
Created attachment 11414 [details]
test case

POSIX 2008 says, in 2.13.3 Patterns Used for Filename Expansion:

<http://pubs.opengroup.org/onlinepubs/9699919799.2008edition/>

>   The <slash> character in a pathname shall be explicitly matched by using one or more <slash> characters in the pattern; it shall neither be matched by the <asterisk> or <question-mark> special characters nor by a bracket expression. <slash> characters in the pattern shall be identified before bracket expressions; thus, a <slash> cannot be included in a pattern bracket expression used for filename expansion. If a <slash> character is found following an unescaped <left-square-bracket> character before a corresponding <right-square-bracket> is found, the open bracket shall be treated as an ordinary character. For example, the pattern "a[b/c]d" does not match such pathnames as abd or a/d. It only matches a pathname of literally a[b/c]d.

But fnmatch (with FNM_PATHNAME) matches pattern `a[b/c]d` with string `abd` not `a[b/c]d`.
Comment 1 Andreas Schwab 2018-11-27 09:52:34 UTC
fnmatch references 2.13.3 only for the interpretation of FNM_PERIOD, not for FNM_PATHNAME.  The description of FNM_PATHNAME in fnmatch only repeats the first sentence from 2.13.3#1, not the rest of it.
Comment 2 Cyker Way 2018-11-27 18:29:10 UTC
(In reply to Andreas Schwab from comment #1)
> fnmatch references 2.13.3 only for the interpretation of FNM_PERIOD, not for
> FNM_PATHNAME.  The description of FNM_PATHNAME in fnmatch only repeats the
> first sentence from 2.13.3#1, not the rest of it.

So this is a word in progress? But `man 3 fnmatch` says it is conforming to POSIX 2008 while in fact it doesn't:

>   CONFORMING TO
>
>      POSIX.1-2001, POSIX.1-2008, POSIX.2.  The FNM_FILE_NAME,
>      FNM_LEADING_DIR, and FNM_CASEFOLD flags are GNU extensions.

<http://man7.org/linux/man-pages/man3/fnmatch.3.html>
Comment 3 Andreas Schwab 2018-11-27 19:03:40 UTC
What do you mean with "word in progress"?
Comment 4 Cyker Way 2018-11-28 01:45:31 UTC
typo: work* in progress. I mean the current implementation obviously doesn't conform to posix 2008 but its man page says so. Either there's work in progress or there's a bug.
Comment 5 Andreas Schwab 2018-11-28 09:08:49 UTC
There is no non-conformance.
Comment 6 Cyker Way 2018-11-29 01:14:59 UTC
Well let me be slightly verbose:

POSIX has **two** requirements in 2.13.3#1:

1. The <slash> character in a pathname shall be explicitly matched by using one or more <slash> characters in the pattern; it shall neither be matched by the <asterisk> or <question-mark> special characters nor by a bracket expression.

2. If a <slash> character is found following an unescaped <left-square-bracket> character before a corresponding <right-square-bracket> is found, the open bracket shall be treated as an ordinary character. For example, the pattern "a[b/c]d" does not match such pathnames as abd or a/d. It only matches a pathname of literally a[b/c]d.

The glibc FNM_PATHNAME implementation only conforms to the first requirement, not the second one. I think you are aware of this issue because you said:

>   The description of FNM_PATHNAME in fnmatch only repeats the first sentence from 2.13.3#1, not the rest of it.

So the glibc FNM_PATHNAME implementation only conforms to **its own man page** (which in turn only partially documents POSIX requirements), not **POSIX** itself, and is violating the very example provided by POSIX. Put it another way, the glibc implementation, together with its documentation, does not fully conform to POSIX (even if glibc implementation conforms to its own documentation). Users, when using a fully POSIX-compliant library, would expect pathname pattern "a[b/c]d" to match string "a[b/c]d" not "abd", but glibc is producing the opposite.

To summarize what I said:

    [glibc impl] --conform-> [glibc doc] --non-conform-> [posix]

Therefore:

    [glibc impl] --non-conform-> [posix]

You could say glibc FNM_PATHNAME, by design, is not an implementation of 2.13.3#1, but then I still don't see where 2.13.3#1 is fully implemented in glibc. In that case a POSIX functionality is missing so the implementation is still not POSIX-compliant.
Comment 7 Andreas Schwab 2018-11-29 10:12:51 UTC
The fnmatch description is coming from POSIX, which does _not_ refer to 2.13.3#1.
Comment 8 Cyker Way 2018-11-29 12:39:59 UTC
POSIX fnmatch function page <http://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/fnmatch.html> doesn't mention 2.13.3#1 directly, but it clearly referred to both 2.13.1 and 2.13.2.

Then, the first sentence of 2.13.3 says:

The rules described so far in Patterns Matching a Single Character and Patterns Matching Multiple Characters **are qualified by the following rules that apply** when pattern matching notation is used for filename expansion...

This essentially means the rules of 2.13.1 and 2.13.2 are limited by additional requirements that must be satisfied for pathname expansion. If you don't satisfy these additional requirements, then you are not conforming to these rules (and POSIX in general). So your argument holds only if you claim FNM_PATHNAME is not for pathname expansion, which IMHO doesn't have much ground. I don't flip this thread as UNCONFIRMED again, but I still strongly believe this is indeed a bug.
Comment 9 Andreas Schwab 2018-11-29 12:54:53 UTC
There are references to 2.13.1 and 2.13.2, and to 2.13.3#2 in the FNM_PERIOD.  There is _no_ reference to 2.13.3#1.
Comment 10 Cyker Way 2018-11-29 13:08:58 UTC
>   There is _no_ reference to 2.13.3#1.

It doesn't have to be directly referenced to apply. It automatically applies wherever it says it shall apply. If you are not conforming to any term in POSIX, then you are not conforming to POSIX.
Comment 11 Andreas Schwab 2018-11-29 13:54:20 UTC
fnmatch is not about filename expansion.
Comment 12 Cyker Way 2018-11-29 15:38:15 UTC
>   So your argument holds only if you claim FNM_PATHNAME is not for pathname expansion.

Well, you finally claimed it. But then what would FNM_PATHNAME be designed for?

POSIX says:

<http://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/fnmatch.html>

>   With the FNM_PATHNAME flag, fnmatch() does match pathnames...

Thus it's clear FNM_PATHNAME is for matching pathnames. Now your ultimate defense is: pathname matching is not pathname expansion.

Sounds logical :/