Understanding pathconf vs. PATH_MAX / _PC_PATH_MAX

Wed Jan 3 13:19:00 GMT 2018

On 03/01/2018 10:02, Sebastian M. Ernst wrote:
> Hi everyone,
> 
> I am building a relatively simple FUSE-based filesystem mostly as an
> exercise for now. In this context, I am trying to wrap my head around
> PATH_MAX and _PC_PATH_MAX (primarily on Linux). I'd like to seek some
> clarification on how to deal with those two correctly in the context of
> POSIX compliance.
> 
> This e-mail / question is a "continuation" of a thread started on
> fuse-devel [0]. After digging deeper into the subject, glibc appears to
> be the "next right place" to look for an answer.
> 
> Please correct me if I am wrong. My understanding so far is that while
> PATH_MAX is an operating-system-wide constant (e.g. 4096 on Linux,
> specified in limits.h [1]), an individual file system (mounted
> somewhere) can return a different (e.g. smaller) value through
> _PC_PATH_MAX when probed with the pathconf or fpathconf system calls.
> Unfortunately, I could not locate any authoritative reference other than
> forum comments [2]. The POSIX standards are not very detailed as far as
> I can see [3], but I could be looking for this information at the wrong
> spot.
> 
> Due to the implementation and concept of my filesystem, the maximum path
> length within it is effectively 2 bytes shorter than whatever PATH_MAX
> is. When I test my filesystem for POSIX compliance with "pjdfstest" [4],
> I see lots of errors popping up related to ENAMETOOLONG as one would
> expect. Looking at the tests' code, one can see that the tests' init
> routine calls pathconf in the tested directory (i.e. the tested
> filesystem) [5] for retrieving the value of _PC_PATH_MAX (which is equal
> to PATH_MAX, 4096). I can "fix" most of my breaking tests related to
> maximum path lengths by subtracting 2 from the value returned by
> pathconf (changing this line [6] into "path_max=$((path_max-3))").
> 
> The above behavior begs the question whether my filesystem could
> influence the return value of pathconf when asked for the value of
> _PC_PATH_MAX. It would allow it to return the correct value and software
> running on top of it (like pjdfstest) could correctly determine the
> maximum path length. From what I gather, it is usually recommended when
> writing software to check for the maximum path length (and maximum name
> length) within individual directories at run-time instead of hard-coding
> PATH_MAX into the software in one way or the other. For this
> recommendation to make sense it would require a [FUSE] filesystem to
> return the correct value.
> 
> The maintainer of libfuse confirmed that FUSE does not currently have a
> way of allowing to customize the value of _PC_PATH_MAX as returned by
> pathconf (or just any other way of informing software running on top of
> a [FUSE] filesystem about the correct maximum path length), although it
> should probably exist [for being POSIX compliant].
> 
> Digging a little bit deeper, I found another discussion / answer [7]
> which directed my attention to glibc. On Linux, pathconf is offered
> through glibc (as opposed to by the kernel, as I had expected). Looking
> at the glibc source, on Linux it appears to default to an implementation
> in /sysdeps/posix/pathconf.c [8]. (Am I correct in this assertion?)
> 
> If the mentioned implementation is asked for _PC_REC_XFER_ALIGN for
> instance [9], it calls __statvfs64 (i.e. into the vfs stack in the
> kernel?) and returns the f_frsize field of the filled out statvfs64
> struct. I'd expect something similar for _PC_PATH_MAX, but pathconf
> apparently just returns the hard-coded value of PATH_MAX [10]. This
> makes some sense as the statvfs64 struct type lacks a field for a
> maximum path length [11] (while, interestingly, there is a field for a
> maximum name length, f_namemax), so statvfs simply does not have a way
> for returning such a value. Why is that? Or is it just the wrong place
> to look for it?
> 
> Another thing which makes my curious is a comment in the implementation
> for returning _PC_SYMLINK_MAX [12]. It states: "In general there are no
> limits. If a system has one it should overwrite this case." The comment
> suggests that a "system" (i.e. an operating system [kernel], not a
> filesystem driver?) can overwrite whatever is returned by pathconf. Or
> is it supposed to be read like "the implementation of pathconf can be
> overwritten"? Is this by any chance related to my interest in
> customizing _PC_PATH_MAX?
> 
> Thanks for any help and suggestions.

Indeed for _PC_PATH_MAX linux implementation 
(sysdeps/unix/sysv/linux/{f}pathconf.c) routes it to default posix one 
(sysdeps/posix/{f}pathconf.c) which in turn returns the constant PATH_MAX 
(4096 on Linux).

My understanding is PATH_MAX is defined as the constant mainly because for
Linux is highly dependent of the underlying filesystem , with some not
imposing a hard limit (although they do pose filename length limits). And
following manual guidelines [1], GLIBC usually does not impose such limits
on file operations (only for implementation where such limits is not really
feasible or incurs in potential other issues, as for execve family for
instance).

If you check Linux pathconf (sysdeps/unix/sysv/linux/pathconf.c) it does have
some filesystem specific code for _PC_LINK_MAX (__statfs_link_max). However
this solution is cumbersome and not quite scalable: it requires add logic
for each new filesystem add in Linux kernel (if it differ from LINUX_LINK_MAX)
with potentially extra logic to distinguish to other type (distinguish_extX
for instance).

Now for POSIX compliant system it is indeed expected that it supports at
minimum a value of PATH_MAX (which _POSIX_PATH_MAX being a minimum value).
I think it would be feasible to query it from underlying operational system
if there is consistent way across all filesystem (I haven't check it), however
I am not sure how compliant would be a FS that returns a value *less* than
PATH_MAX.

[1] https://www.gnu.org/software/libc/manual/html_node/Limits-for-Files.html

> 
> Regards,
> Sebastian
> 
> 
> [0]
> https://sourceforge.net/p/fuse/mailman/fuse-devel/thread/e4018f6a-b39c-5045-de99-662a680ae00c%40pleiszenburg.de/#msg36175832
> [1]
> https://github.com/torvalds/linux/blob/ead751507de86d90fa250431e9990a8b881f713c/include/uapi/linux/limits.h
> [2]
> https://stackoverflow.com/questions/34278283/how-can-i-supply-pathconf-with-fuse
> [3] http://pubs.opengroup.org/onlinepubs/9699919799/functions/fpathconf.html
> [4] https://github.com/pjd/pjdfstest
> [5]
> https://github.com/pjd/pjdfstest/blob/9f74a0dca5822388d84fc3f914cc296d93c46213/tests/misc.sh#L150
> [6]
> https://github.com/pjd/pjdfstest/blob/9f74a0dca5822388d84fc3f914cc296d93c46213/tests/misc.sh#L152
> [7] https://stackoverflow.com/a/16973900/1672565
> [8]
> https://github.com/bminor/glibc/blob/09533208febe923479261a27b7691abef297d604/sysdeps/posix/pathconf.c#L30
> [9]
> https://github.com/bminor/glibc/blob/09533208febe923479261a27b7691abef297d604/sysdeps/posix/pathconf.c#L191
> [10]
> https://github.com/bminor/glibc/blob/09533208febe923479261a27b7691abef297d604/sysdeps/posix/pathconf.c#L89
> [11]
> https://github.com/bminor/glibc/blob/09533208febe923479261a27b7691abef297d604/bits/statvfs.h#L59
> [12]
> https://github.com/bminor/glibc/blob/09533208febe923479261a27b7691abef297d604/sysdeps/posix/pathconf.c#L215
> 