[PATCH v4 2/2] io: Use gnulib fts implementation (BZ 22944, BZ 20331)
Collin Funk
collin.funk1@gmail.com
Wed Apr 1 04:19:51 GMT 2026
Hi Adhemerval,
Adhemerval Zanella <adhemerval.zanella@linaro.org> writes:
> This patch synchronizes the glibc fts implementation with the latest
> version from gnulib (as of 2026-02-16).
>
> The primary motivation is to address limitations in the legacy glibc
> implementation, most notably BZ 22944, where fts fails with an
> ENAMETOOLONG error when traversing very long paths or deeply nested
> directory trees. The gnulib implementation dynamically reallocates
> path buffers and uses openat/fchdir optimizations, effectively
> lifting the MAXPATHLEN limitation.
>
> The gnulib implementation also added extra features, which are
> used by different GNU projects (coreutils, diffutils):
diffutils doesn't use fts. The main ones I know of, other than
coreutils, are findutils and grep.
> * FTS_TIGHT_CYCLE_CHECK: used to enable a strict, immediate
> cycle-detection algorithm during a file system traversal. This is
> done internally using a hash table: every time the traversal enters
> a directory, it records the directory's device and inode (dev/ino)
> pair in the hash table, and before entering any directory, fts
> checks the hash table.
>
> * FTS_CWDFD: instead of actually changing the process's current
> working directory, it maintains a virtual current working directory
> using file descriptors. The file descriptor is store at the
> fts_cwd_fd field and all subsequent file operations are performed
> relative to this file descriptor using *at functions.
>
> * FTS_DEFER_STAT: performance-oriented flag that instructs the file
> tree traversal engine to delay fetching file metadata. When the
> flag is used, fts skips the immediate stat call. Instead, it marks
> the entry with a special internal state (FTS_NSOK and
> FTS_STAT_REQUIRED). The actual stat call is pushed down the line
> and executed by fts_read right before the application actually
> accesses the entry.
>
> * FTS_VERBATIM: fts_open aaccept and use the path strings exactly as
> they were provided in the arguments array without slash trimming.
>
> * FTS_MOUNT: it restrict the file tree walk to a single file system.
>
> Hopefully,it would allow some GNU projects to use the glibc
> implementation instead of pulling the gnulib one.
Missing space after the comma.
>
> It requires some changes to keep compatibility, compared to gnulib:
>
> * The new required fields are added at the end of FTS structure, and
> the new FTS flags are adjusted to avoid change FTS_NAMEONLY/FTS_STOP
> (even though they are marked as private).
>
> * The FTSENT uses a flexible array (fts_name), so two adjustments are
> required: the two new members (fts_fts and fts_dirp) are place
> *before* the struct and the fts_statp is now always allocated and
> accounted (the gnulib implementation uses a awalys allocated member).
>
> Checked on x86_64-linux-gnu and i686-linux-gnu.
> --
> Changes from v3:
> * Move stdc_rotate_right to a different patch, along with tests from
> gnulib.
> Changes from v2:
> * Remove bitrotate.h and replace with internal stdc_rotate_right
> implementation.
> Changes from v1:
> * Move next-prime.c, hash.c to io subfolder, and hash.h and next-prive.h
> to include/.
> ---
> SHARED-FILES | 16 +
> dirent/scandirat.c | 3 +-
> dirent/scandirat64.c | 3 +-
> include/assure.h | 57 +
> include/dirent.h | 3 +-
> include/flexmember.h | 77 +
> include/fts.h | 7 +
> include/hash.h | 331 ++++
> include/next-prime.h | 47 +
> include/xalloc-oversized.h | 65 +
> io/Makefile | 2 +
> io/cycle-check.c | 85 +
> io/cycle-check.h | 70 +
> io/dev-ino.h | 44 +
> io/fts-cycle.c | 162 ++
> io/fts.c | 2902 ++++++++++++++++++++---------
> io/fts.h | 126 +-
> io/fts64-time64.c | 25 +-
> io/fts64.c | 5 +-
> io/i-ring.c | 69 +
> io/same-inode.h | 106 ++
> io/tst-fts-bz22944.c | 100 +
> io/tst-fts-newflags.c | 234 +++
> io/tst-fts.c | 2 +
> misc/Makefile | 2 +
> misc/hash.c | 1045 +++++++++++
> misc/next-prime.c | 64 +
> sysdeps/mach/hurd/opendir.c | 7 +-
> sysdeps/unix/sysv/linux/opendir.c | 16 +-
> 29 files changed, 4737 insertions(+), 938 deletions(-)
> create mode 100644 include/assure.h
> create mode 100644 include/flexmember.h
> create mode 100644 include/hash.h
> create mode 100644 include/next-prime.h
> create mode 100644 include/xalloc-oversized.h
> create mode 100644 io/cycle-check.c
> create mode 100644 io/cycle-check.h
> create mode 100644 io/dev-ino.h
> create mode 100644 io/fts-cycle.c
> create mode 100644 io/i-ring.c
> create mode 100644 io/same-inode.h
> create mode 100644 io/tst-fts-bz22944.c
> create mode 100644 io/tst-fts-newflags.c
> create mode 100644 misc/hash.c
> create mode 100644 misc/next-prime.c
The files are mostly unchanged from Gnulib, outside of some namespacing
macros and such.
> diff --git a/io/fts.h b/io/fts.h
> index b496e75cee..cc30d282d3 100644
> --- a/io/fts.h
> +++ b/io/fts.h
> @@ -52,7 +52,29 @@
>
> #include <features.h>
> #include <sys/types.h>
> +#include <sys/stat.h>
> +#include <dirent.h>
> +#include <stdbool.h>
>
> +enum { I_RING_SIZE = 4 };
> +
> +/* When ir_empty is true, the ring is empty.
> + Otherwise, ir_data[B..F] are defined, where B..F is the contiguous
> + range of indices, modulo I_RING_SIZE, from back to front, inclusive.
> + Undefined elements of ir_data are always set to ir_default_val.
> + Popping from an empty ring aborts.
> + Pushing onto a full ring returns the displaced value.
> + An empty ring has F==B and ir_empty == true.
> + A ring with one entry still has F==B, but now ir_empty == false. */
> +struct I_ring
> +{
> + int ir_data[I_RING_SIZE];
> + int ir_default_val;
> + unsigned int ir_front;
> + unsigned int ir_back;
> + bool ir_empty;
> +};
> +typedef struct I_ring I_ring;
Can we move the I_ring struct, typedef, and enum under the "__*"
namespace? I would like to avoid duplicate/conflicting definitions if
Gnulib ever needs to make changes.
> diff --git a/io/tst-fts-bz22944.c b/io/tst-fts-bz22944.c
> new file mode 100644
> index 0000000000..41bcd5f597
> --- /dev/null
> +++ b/io/tst-fts-bz22944.c
> @@ -0,0 +1,100 @@
> +/* Check if fts does not fail with very long paths.
I think "Check that" makes this sentence a bit more clear. The test
itself is good.
> +enum { nested_depth = 150 };
> +static const char dir_name[] = { [0 ... 254] = 'A', '\0' };
Interesting, I am not used to seeing that syntax. But I guess we use it
elsewhere quite often.
> diff --git a/io/tst-fts-newflags.c b/io/tst-fts-newflags.c
> new file mode 100644
> index 0000000000..49cd630f3a
> --- /dev/null
> +++ b/io/tst-fts-newflags.c
This test looks good as well.
> +/* FTS_VERBATIM: eEnsures paths are passed through without having trailing
s/eE/E/
Collin
More information about the Libc-alpha
mailing list