copying a million tiny files?
Thu Nov 1 06:52:00 GMT 2007
Gary R. Van Sickle wrote:
> I have a similar situation to the OP (copying many thousands of small
> files over a fairly slow link), and actually timed using XCOPY vs.
> Cygwin methods (cp in my case). It didn't make a significant
> difference. Ultimately what I think you run into in these sorts of
> situations is that you bump up against the slowness of the link (or
> physical disk) because, POSIX emulation or not, all your caches do is
I think your situation is vastly different. Granted copying over a
network connection will be soo much slower that the additional overhead
of Cygwin emulation fades away. But the OP was talking about copying
between to supposedly local disks. With the network link out of the way
the overhead of Cygwin emulation could become significant.
As for your cp across a link... At one time I was using cp to copy our
release images (about 9 files of about 50-150 meg mind you) across a WAN
link (from Santa Clara, California -> Shanghai, China). The link was
real slow and it was talking ~30-45 minutes per image IIRC!
Then I changed it to an ncftp and the time per image dropped down to ~5
Moral of the story? SMB is perhaps the worse way to transfer data across
Andrew DeFaria <http://defaria.com>
I used to have an open mind but my brains kept falling out.
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
More information about the Cygwin