[PATCH] io: Refactor {n}ftw to use fts for stack safety and conformance (BZ 33882)

[Adhemerval Zanella Netto]

On 16/02/26 09:51, Adhemerval Zanella Netto wrote:
> 
> 
> On 15/02/26 13:51, Florian Weimer wrote:
>> * Adhemerval Zanella Netto:
>>
>>> On 13/02/26 19:14, Collin Funk wrote:
>>>> Adhemerval Zanella Netto <adhemerval.zanella@linaro.org> writes:
>>>>>
>>>>> However it does way better memory usage for the pathological cases. Using the
>>>>> nested 5000 directories from tst-nftw-bz33882 massif for master shows:
>>>> [...]
>>>>> And I do not if we can implement a feasible {n}ftw with recursion *without*
>>>>> imposing some path limits (which is an arbitrary limits that conflict with the
>>>>> GNU Coding Standards).
>>>>>
>>>>> Maybe we can optimize it better by reimplement a FTS without some extra
>>>>> requirement, but this will also duplicate code.
>>>>
>>>> FWIW, my opinion is that the ability to visit arbitrary depths is more
>>>> important than the performance here.
>>>>
>>>> Gnulib uses a slightly modified implementation of fts, e.g., it uses
>>>> openat and avoids f?chdir, but both versions manage memory very
>>>> efficiently and can visit arbitrary depths. This is quite important for
>>>> GNU coreutils, for obvious reasons. Other implementations tend to impose
>>>> arbitrary limits and/or overflow the stack.
>>>
>>> As I added on BZ#33882, the glibc FTS implementation is subpar because it is 
>>> constraint by the ABI (we can not change size, member alignment, or the 
>>> semantics of existing members of FTS and FTSENT without all the extra burden 
>>> of symbol versioning). 
>>
>> I don't think applications can meaningfully allocate the FTS type or
>> copy it, so we could add hidden members (with an internal umbrella
>> struct) that are not present on the external interface?
> 
> The main issue is the _ftsent, where is uses a flexible array for the
> file name and the gnulib added some internal member over the time.  So
> we can no easily add internal fields without breaking compatibility.

I think we can internally allocate it as:

 struct wrapper_ftsent
 {
   /* required extra fields */
   FTSENT fsent;
 };

and then accessing the fields when required assuming this layout.  It does
complicate a bit the code, but it should be doable.

> 
>>
>> The other issue is that if the full pathname is longer than PATH_MAX,
>> it can only be opened piecewise on Linux, using relative pathnames.
>>
> 
> I think the gnulib implements does handle that in a optimized way.