Tuning ls, ls -l, find ., find . -ls performance for very large dirs (60000+ files/dir)

[Brian Inglis]

On 08/04/2025 19:05, Mark Liam Brown via Cygwin wrote:
> Are there tuning variables to improve ls, ls -l, find ., find . -ls
> performance for very large dirs?
>
> If we have a SMB dir with 60000+ entries a simple ls -l can take MANY
> minutes (22+mins), while cmd.exe dir just floods the terminal with
> results immediately.

I just remembered noticing my /var/log/ entries jumped looking at my last Cygwin 
upgrade log, so before I clean it up into subdirs, let's try some stuff!

If you just want a directory name dump, use \ls -U|--sort=none or \ls -f which 
is like -aU; command prefix \ overrides the usual aliases like -CF --color=auto:

$ time \ls -1U /var/log/ | \wc -lwmcL
   54049   54049 1885787 1885787      38

real	0m0.336s
user	0m0.046s
sys	0m0.295s
$ time \ls -1f /var/log/ | \wc -lwmcL
   54051   54051 1885792 1885792      38

real	0m0.347s
user	0m0.140s
sys	0m0.232s

even with -l, it's only a few seconds:

$ time \ls -lU /var/log/ | \wc -lwmcL
   54050  486455 4966716 4966716     102

real	0m15.891s
user	0m1.421s
sys	0m14.560s
$ time \ls -lf /var/log/ | \wc -lwmcL
   54052  486473 4966835 4966835     102

real	0m15.858s
user	0m1.405s
sys	0m14.374s

and skipping user and group lookup does not save anything:

$ time \ls -gloU /var/log/ | \wc -lwmcL
   54050  378357 3777638 3777638      80

real	0m15.699s
user	0m1.562s
sys	0m14.155s
$ time \ls -gGlf /var/log/ | \wc -lwmcL
   54052  378371 3777713 3777713      80

real	0m15.908s
user	0m1.389s
sys	0m14.498s

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retrancher  but when there is no more to cut
                                 -- Antoine de Saint-Exupéry