This is the mail archive of the
mailing list for the Cygwin project.
Re: Rsync performance increase through buffering (cygwin/ssh solution?)
- From: Jim Kleckner <jek-news3 at kleckner dot net>
- To: cygwin at cygwin dot com
- Date: Thu, 12 Dec 2002 11:30:12 -0800
- Subject: Re: Rsync performance increase through buffering (cygwin/ssh solution?)
Below is a description of a buffering speed improvement.
The original message with complete patch for performance can be found here:
This may explain the horrendous performance of rsync
with ssh under cygwin. Perhaps the way that cygwin
emulates these calls. It is especially bad under the
circumstances mentioned here - with a large sync of data
where the files are identical (-I).
The hang fix has been identified elsewhere in cygwin:
Check out these links:
patched rsync for hang under cygwin:
email@example.com (Craig Barratt) wrote in message
> I've been studying the read and write buffering in rsync and it turns
> out most I/O is done just a couple of bytes at a time. This means there
> are lots of system calls, and also most network traffic comprises lots
> of small packets. The behavior is most extreme when sending/receiving
> file deltas of identical files.
> The main case where I/O is buffered is writes from the server (when
> io multiplexing is on). These are usually buffered in 4092 byte
> chunks with a 4 byte header. However, reading of these packets is
> usually unbuffered, and writes from the client are generally not
> buffered. For example: when receiving 1st phase checksums (6 bytes
> per block), 2 reads are done: one of 4 bytes and one of 2 bytes,
> meaning there are 4 system calls (select/read/select/read) per 6
> bytes of checksum data).
> One cost of this is some performance, but a significant issue is that
> unbuffered writes generate very short (and very many) ethernet packets,
> which means the overhead is quite large on slow network connections.
... see this link for the rest:
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html