[PATCH] io:nftw/ftw:fix stack overflow when large nopenfd [BZ #26353]

[Xiaoming Ni]

On 2020/8/14 19:14, Xiaoming Ni wrote:
> On 2020/8/14 4:32, Paul Eggert wrote:
>> The patch isn't complete, since it doesn't check for integer overflow 
>> when multiplying data.maxdir by sizeof (struct dir_data *), where the 
>> function should also fail with errno == ENOMEM. You can check for 
>> overflow via intprops.h's INT_MULTIPLY_WRAPV (data.maxdir, sizeof 
>> (struct dir_data *)), &x) where x is of type size_t.
>>
> diff --git a/io/ftw.c b/io/ftw.c
> index 8c79d29a9e..094aada50c 100644
> --- a/io/ftw.c
> +++ b/io/ftw.c
> @@ -643,18 +643,32 @@ ftw_startup (const char *dir, int is_nftw, void 
> *func, int descriptors,
>         __set_errno (ENOENT);
>         return -1;
>       }
> +  if (descriptors > getdtablesize())
> +    {
> +      __set_errno (EINVAL);
> +      return -1;
> +    }
> linux/include/uapi/linux/fs.h:38:#define INR_OPEN_MAX 4096   /* Hard 
> limit for nfile rlimits */
> 
> When data.maxdir is less than getdtablesize(), is there still a 
> possibility that integer overflow occurs in data.maxdir * sizeof (struct 
> dir_data *)?
> 

on linux:
The maximum number of process file handles is sysctl_nr_open.
/proc/sys/fs/nr_open
The maximum value is sysctl_nr_open_max.

  fs/file.c :
#define __const_min(x, y) ((x) < (y)? (x): (y))
unsigned int sysctl_nr_open_max = __const_min(INT_MAX, 
~(size_t)0/sizeof(void *)) & -BITS_PER_LONG;

On a 32 - bit machine,
BITS_PER_LONG is 32
INT_MAX  is 0x7fffffff
SIZE_MAX is 0xffffffff
sysctl_nr_open_max is 0x3fffffe0
sysctl_nr_open_max * sizeof (struct dir_data *)) is 0xffffff80
The value is greater than INT_MAX but less than SIZE_MAX.
No overflow occurs.

On a 64-bit machine
BITS_PER_LONG is 64
INT_MAX  is 0x7fffffff
SIZE_MAX is 0xffffffffffffffff
sysctl_nr_open_max is 0x7fffffc0
sysctl_nr_open_max * sizeof (struct dir_data *)) is 0x3ffffff00
The value is greater than INT_MAX but less than SIZE_MAX. No overflow 
occurs.

Thanks
Xiaoming Ni