[PATCH v4 2/2] io: Use gnulib fts implementation (BZ 22944, BZ 20331)
Adhemerval Zanella Netto
adhemerval.zanella@linaro.org
Thu Apr 2 19:31:22 GMT 2026
On 01/04/26 01:19, Collin Funk wrote:
> Hi Adhemerval,
>
> Adhemerval Zanella <adhemerval.zanella@linaro.org> writes:
>
>> This patch synchronizes the glibc fts implementation with the latest
>> version from gnulib (as of 2026-02-16).
>>
>> The primary motivation is to address limitations in the legacy glibc
>> implementation, most notably BZ 22944, where fts fails with an
>> ENAMETOOLONG error when traversing very long paths or deeply nested
>> directory trees. The gnulib implementation dynamically reallocates
>> path buffers and uses openat/fchdir optimizations, effectively
>> lifting the MAXPATHLEN limitation.
>>
>> The gnulib implementation also added extra features, which are
>> used by different GNU projects (coreutils, diffutils):
>
> diffutils doesn't use fts. The main ones I know of, other than
> coreutils, are findutils and grep.
>
>> * FTS_TIGHT_CYCLE_CHECK: used to enable a strict, immediate
>> cycle-detection algorithm during a file system traversal. This is
>> done internally using a hash table: every time the traversal enters
>> a directory, it records the directory's device and inode (dev/ino)
>> pair in the hash table, and before entering any directory, fts
>> checks the hash table.
>>
>> * FTS_CWDFD: instead of actually changing the process's current
>> working directory, it maintains a virtual current working directory
>> using file descriptors. The file descriptor is store at the
>> fts_cwd_fd field and all subsequent file operations are performed
>> relative to this file descriptor using *at functions.
>>
>> * FTS_DEFER_STAT: performance-oriented flag that instructs the file
>> tree traversal engine to delay fetching file metadata. When the
>> flag is used, fts skips the immediate stat call. Instead, it marks
>> the entry with a special internal state (FTS_NSOK and
>> FTS_STAT_REQUIRED). The actual stat call is pushed down the line
>> and executed by fts_read right before the application actually
>> accesses the entry.
>>
>> * FTS_VERBATIM: fts_open aaccept and use the path strings exactly as
>> they were provided in the arguments array without slash trimming.
>>
>> * FTS_MOUNT: it restrict the file tree walk to a single file system.
>>
>> Hopefully,it would allow some GNU projects to use the glibc
>> implementation instead of pulling the gnulib one.
>
> Missing space after the comma.
Ack.
>
>>
>> It requires some changes to keep compatibility, compared to gnulib:
>>
>> * The new required fields are added at the end of FTS structure, and
>> the new FTS flags are adjusted to avoid change FTS_NAMEONLY/FTS_STOP
>> (even though they are marked as private).
>>
>> * The FTSENT uses a flexible array (fts_name), so two adjustments are
>> required: the two new members (fts_fts and fts_dirp) are place
>> *before* the struct and the fts_statp is now always allocated and
>> accounted (the gnulib implementation uses a awalys allocated member).
>>
>> Checked on x86_64-linux-gnu and i686-linux-gnu.
>> --
>> Changes from v3:
>> * Move stdc_rotate_right to a different patch, along with tests from
>> gnulib.
>> Changes from v2:
>> * Remove bitrotate.h and replace with internal stdc_rotate_right
>> implementation.
>> Changes from v1:
>> * Move next-prime.c, hash.c to io subfolder, and hash.h and next-prive.h
>> to include/.
>> ---
>> SHARED-FILES | 16 +
>> dirent/scandirat.c | 3 +-
>> dirent/scandirat64.c | 3 +-
>> include/assure.h | 57 +
>> include/dirent.h | 3 +-
>> include/flexmember.h | 77 +
>> include/fts.h | 7 +
>> include/hash.h | 331 ++++
>> include/next-prime.h | 47 +
>> include/xalloc-oversized.h | 65 +
>> io/Makefile | 2 +
>> io/cycle-check.c | 85 +
>> io/cycle-check.h | 70 +
>> io/dev-ino.h | 44 +
>> io/fts-cycle.c | 162 ++
>> io/fts.c | 2902 ++++++++++++++++++++---------
>> io/fts.h | 126 +-
>> io/fts64-time64.c | 25 +-
>> io/fts64.c | 5 +-
>> io/i-ring.c | 69 +
>> io/same-inode.h | 106 ++
>> io/tst-fts-bz22944.c | 100 +
>> io/tst-fts-newflags.c | 234 +++
>> io/tst-fts.c | 2 +
>> misc/Makefile | 2 +
>> misc/hash.c | 1045 +++++++++++
>> misc/next-prime.c | 64 +
>> sysdeps/mach/hurd/opendir.c | 7 +-
>> sysdeps/unix/sysv/linux/opendir.c | 16 +-
>> 29 files changed, 4737 insertions(+), 938 deletions(-)
>> create mode 100644 include/assure.h
>> create mode 100644 include/flexmember.h
>> create mode 100644 include/hash.h
>> create mode 100644 include/next-prime.h
>> create mode 100644 include/xalloc-oversized.h
>> create mode 100644 io/cycle-check.c
>> create mode 100644 io/cycle-check.h
>> create mode 100644 io/dev-ino.h
>> create mode 100644 io/fts-cycle.c
>> create mode 100644 io/i-ring.c
>> create mode 100644 io/same-inode.h
>> create mode 100644 io/tst-fts-bz22944.c
>> create mode 100644 io/tst-fts-newflags.c
>> create mode 100644 misc/hash.c
>> create mode 100644 misc/next-prime.c
>
> The files are mostly unchanged from Gnulib, outside of some namespacing
> macros and such.
Yeah, I tried to avoid changes that would not be backportable to gnulib.
>
>> diff --git a/io/fts.h b/io/fts.h
>> index b496e75cee..cc30d282d3 100644
>> --- a/io/fts.h
>> +++ b/io/fts.h
>> @@ -52,7 +52,29 @@
>>
>> #include <features.h>
>> #include <sys/types.h>
>> +#include <sys/stat.h>
>> +#include <dirent.h>
>> +#include <stdbool.h>
>>
>> +enum { I_RING_SIZE = 4 };
>> +
>> +/* When ir_empty is true, the ring is empty.
>> + Otherwise, ir_data[B..F] are defined, where B..F is the contiguous
>> + range of indices, modulo I_RING_SIZE, from back to front, inclusive.
>> + Undefined elements of ir_data are always set to ir_default_val.
>> + Popping from an empty ring aborts.
>> + Pushing onto a full ring returns the displaced value.
>> + An empty ring has F==B and ir_empty == true.
>> + A ring with one entry still has F==B, but now ir_empty == false. */
>> +struct I_ring
>> +{
>> + int ir_data[I_RING_SIZE];
>> + int ir_default_val;
>> + unsigned int ir_front;
>> + unsigned int ir_back;
>> + bool ir_empty;
>> +};
>> +typedef struct I_ring I_ring;
>
> Can we move the I_ring struct, typedef, and enum under the "__*"
> namespace? I would like to avoid duplicate/conflicting definitions if
> Gnulib ever needs to make changes.
Alright.
>
>> diff --git a/io/tst-fts-bz22944.c b/io/tst-fts-bz22944.c
>> new file mode 100644
>> index 0000000000..41bcd5f597
>> --- /dev/null
>> +++ b/io/tst-fts-bz22944.c
>> @@ -0,0 +1,100 @@
>> +/* Check if fts does not fail with very long paths.
>
> I think "Check that" makes this sentence a bit more clear. The test
> itself is good.
Ack.
>
>> +enum { nested_depth = 150 };
>> +static const char dir_name[] = { [0 ... 254] = 'A', '\0' };
>
> Interesting, I am not used to seeing that syntax. But I guess we use it
> elsewhere quite often.
Afaik Designated Initializer Ranges is a gcc extension, it can be pretty
useful.
>
>> diff --git a/io/tst-fts-newflags.c b/io/tst-fts-newflags.c
>> new file mode 100644
>> index 0000000000..49cd630f3a
>> --- /dev/null
>> +++ b/io/tst-fts-newflags.c
>
> This test looks good as well.
>
>> +/* FTS_VERBATIM: eEnsures paths are passed through without having trailing
>
> s/eE/E/
Ack.
>
> Collin
More information about the Libc-alpha
mailing list