incredibly slow file listing script on windoze 7 pro 4 core 64 bit

[Larry Hall (Cygwin)]

On 9/8/2010 9:35 AM, mike marchywka wrote:
> On 9/7/10, Larry Hall (Cygwin)<reply-to-list-only-lh@xxx.com>  wrote:
>> On 9/7/2010 12:05 PM, mike marchywka wrote:
>>> this takes a few minutes on old debian machine, taking much longer
>>> here on same data-
>>> about 19k file about 24Gb total size.
>>> Windoze finally has better perf stuff but still no help- one core at
>>> 25 pct all kernel
>>> time disk not exactly busy. All the time is in the "ls" loop not the
>>> find command.
>>> Now obbviously I expect the "ls" per file has to make a bunch of OS calls
>>> for
>>> each file but still even with cygwin layer seems a bit much.
>>> Thanks.
>>
>> The problem is that Cygwin has to open all the files to stat when you
>> use the '-l' flag.  Try without the flag to get the lower bound and see if
>> that's better for you.  If so, maybe '-s' will suit your purpose.  Or
>> even 'du'?
>
> Sure, I could find a better way to do it but I was just curious about
> performance diffs. I've got this expensive 'doze laptop with 4 cores and the
> latest OS and all and compared to this old emachines desktop with debian it
> seems a bit slow. Now maybe this is a bit of a contrived case, no one would
> really do this and afterall the new system may be optimized for real-life
> usages and of course cygwin can't speed up obligatory OS calls.

To somewhat sooth your curiousity, Windows (or perhaps it's more accurate
to say NTFS) ain't great with directories with a large number of files.
I expect you would be less than impressed with the performance of of 'dir'
in 'cmd.exe' in the same directory.  That said, you're asking for allot more
work than you may realize when doing the same thing in Cygwin.  In order to
fill in the information you ask for when you use the '-l' flag for 'ls',
Cygwin needs to open and close the files, which takes a good chunk of time
en masse.  The same does not need to happen in Linux/UNIX-land.  So what
you're looking at here, above and beyond the design decisions of Windows,
is some limitations of the emulated environment.  So you can now take some
joy in the knowledge that you have a spanking new system to run
Windows/Cygwin on rather than an old eMachines box. ;-)

> Anyway yes du seems much faster and is quite reasonable :)

Glad it helps.

-- 
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
216 Dalton Rd.                          (508) 893-9889 - FAX
Holliston, MA 01746

_____________________________________________________________________

A: Yes.
> Q: Are you sure?
>> A: Because it reverses the logical flow of conversation.
>>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple