[PATCH v5 2/2] io: Use gnulib fts implementation (BZ 22944, BZ 20331)
Collin Funk
collin.funk1@gmail.com
Sat Apr 11 01:56:46 GMT 2026
Hi Florian,
Florian Weimer <fweimer@redhat.com> writes:
>> Collin Funk <collin.funk1@gmail.com> writes:
>>
>>> It would be helpful to see if ftsent->fts_info before and after the
>>> change. I suspect it may have been returning FTS_D before and is now
>>> returning FTS_NS now. This would explain why it did not pick up
>>> hv_storvsc from under the scsi subdirectory.
>>
>> Oops, I meant FTS_NSOK instead of FTS_NSOK. I think it would be safer
>> for dracut to do something like this since it is using FTS_NOSTAT:
>>
>> /* Get the file type if it could not be determined without stat. */
>> int info = fts_info;
>> if (info == FTS_NSOK)
>> {
>> fts_set (fts, ent, FTS_AGAIN);
>> fts_read (fts);
>> info = ent->fts_info;
>> }
>>
>> I don't think it was ever safe to assume that fts_info would be set to
>> FTS_D (or another file type) when FTS_NOSTAT was used.
>
> Thanks for looking into this. I wonder how common this misuse is. Is
> this another example?
>
> <https://sources.debian.org/src/rpki-client/9.7-1/src/repo.c>
It is difficult to say since I am not very familiar with that program. I
see they do handle the case of FTS_NSOK, but I am not sure if the
behavior is correct or not.
The real issue isn't exactly as I described it, but the general idea is
the same.
Using this test program:
$ cat main.c
#if GNULIB
# include <config.h>
#endif
#include <stdlib.h>
#include <stdio.h>
#if GNULIB
# include "fts_.h"
#else
# include <fts.h>
#endif
static char *
fts_type (FTSENT *ent)
{
switch (ent->fts_info)
{
case FTS_D: return "FTS_D";
case FTS_DC: return "FTS_DC";
case FTS_DEFAULT: return "FTS_DEFAULT";
case FTS_DNR: return "FTS_DNR";
case FTS_DOT: return "FTS_DOT";
case FTS_DP: return "FTS_DP";
case FTS_ERR: return "FTS_ERR";
case FTS_F: return "FTS_F";
case FTS_NS: return "FTS_NS";
case FTS_NSOK: return "FTS_NSOK";
case FTS_SL: return "FTS_SL";
case FTS_SLNONE: return "FTS_SLNONE";
default:
abort ();
}
}
int
main (void)
{
char *const files[] = { "a", NULL };
FTS *fts = fts_open (files, (FTS_COMFOLLOW | FTS_NOCHDIR
| FTS_NOSTAT | FTS_LOGICAL), NULL);
if (! fts)
abort ();
for (FTSENT *ent; (ent = fts_read (fts));)
printf ("%s %s\n", ent->fts_accpath, fts_type (ent));
return 0;
}
And then running this script to compare the output:
$ cat test.sh
#!/bin/sh
rm -rf testdir1
gnulib-tool --create-testdir --dir testdir1 fts
(cd testdir1 && ./configure && make)
mkdir -p a/b
touch a/b/1 a/b/2 a/b/3
gcc -c main.c -o main-glibc.o
gcc -Itestdir1/gllib -Itestdir1 -DGNULIB=1 -c main.c -o main-gnulib.o
gcc -o main-glibc main-glibc.o
gcc -Ltestdir1/gllib -o main-glibc main-glibc.o -lgnu
echo glibc
./main-glibc
echo gnulib
./main-gnulib
Here is what we see:
$ ./test.sh
glibc
a FTS_D
a/b FTS_D
a/b/1 FTS_F
a/b/2 FTS_F
a/b/3 FTS_F
a/b FTS_DP
a FTS_DP
gnulib
a FTS_D
a/b FTS_D
a/b/1 FTS_NSOK
a/b/2 FTS_NSOK
a/b/3 FTS_NSOK
a/b FTS_DP
a FTS_DP
So glibc 2.42, before this change, returned FTS_F in fts_info for the
created files. The Gnulib version returns FTS_NSOK. Assuming it isn't
some other difference causing issues for dracut, I expect that this
branch was taken [1]:
if ((ftsent->fts_info != FTS_F) && (ftsent->fts_info != FTS_SL)) {
log_debug("Ignoring %s", ftsent->fts_accpath);
continue;
}
It might be helpful if the original reporter can enable dracut logs to
see if that message is printed. I would request it from them myself, but
I am not very familiar with dracut.
I do think that it is a bug to not handle FTS_NSOK when using
FTS_NOSTAT.
Also, note that glibc-2.42's behavior is quite poor since it calls stat
on regular files despite the users request not to with FTS_NOSTAT:
$ strace --quiet=exit -e trace='/stat|open' ./main-glibc > /dev/null
[...]
newfstatat(AT_FDCWD, "a", {st_mode=S_IFDIR|0755, st_size=2, ...}, 0) = 0
fstat(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}) = 0
openat(AT_FDCWD, "a", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=2, ...}) = 0
newfstatat(AT_FDCWD, "a/b", {st_mode=S_IFDIR|0755, st_size=6, ...}, 0) = 0
openat(AT_FDCWD, "a/b", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
newfstatat(AT_FDCWD, "a/b/1", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "a/b/2", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
newfstatat(AT_FDCWD, "a/b/3", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Compared to Gnulib:
$ strace --quiet=exit -e trace='/stat|open' ./main-gnulib > /dev/null
[...]
newfstatat(AT_FDCWD, "a", {st_mode=S_IFDIR|0755, st_size=2, ...}, 0) = 0
fstat(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}) = 0
openat(AT_FDCWD, "a", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=2, ...}) = 0
newfstatat(AT_FDCWD, "a/b", {st_mode=S_IFDIR|0755, st_size=6, ...}, 0) = 0
openat(AT_FDCWD, "a/b", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
This optimization is quite important. For example, 'chown' and 'chmod'
do not need to stat the files that they are processing. Using the
information from stat in those programs would likely introduce some
nasty TOCTOU races. Therefore, there is no need to do all this extra
work that glibc does:
$ (cd a/ && seq 1000000 | parallel -m truncate -s 0)
$ strace -c -e trace='/stat' ./main-glibc > /dev/null
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 8.618300 8 1000008 2 newfstatat
0.00 0.000028 5 5 fstat
------ ----------- ----------- --------- --------- ----------------
100.00 8.618328 8 1000013 2 total
Compared to the Gnulib implementation:
$ strace -c -e trace='/stat' ./main-gnulib > /dev/null
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
73.33 0.000011 2 5 2 newfstatat
26.67 0.000004 0 5 fstat
------ ----------- ----------- --------- --------- ----------------
100.00 0.000015 1 10 2 total
Collin
[1] https://github.com/dracutdevs/dracut/blob/5d2bda46f4e75e85445ee4d3bd3f68bf966287b9/src/install/dracut-install.c#L2001-L2004
More information about the Libc-alpha
mailing list