This is the mail archive of the mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Rsync performance increase through buffering (cygwin/ssh solution?)

Below is a description of a buffering speed improvement.
The original message with complete patch for performance can be found here:

This may explain the horrendous performance of rsync
with ssh under cygwin. Perhaps the way that cygwin
emulates these calls. It is especially bad under the
circumstances mentioned here - with a large sync of data
where the files are identical (-I).
The hang fix has been identified elsewhere in cygwin:
Check out these links:
patched rsync for hang under cygwin:

Jim (Craig Barratt) wrote in message news:<at1i0e$1tj3$>...
> I've been studying the read and write buffering in rsync and it turns
> out most I/O is done just a couple of bytes at a time. This means there
> are lots of system calls, and also most network traffic comprises lots
> of small packets. The behavior is most extreme when sending/receiving
> file deltas of identical files.
> The main case where I/O is buffered is writes from the server (when
> io multiplexing is on). These are usually buffered in 4092 byte
> chunks with a 4 byte header. However, reading of these packets is
> usually unbuffered, and writes from the client are generally not
> buffered. For example: when receiving 1st phase checksums (6 bytes
> per block), 2 reads are done: one of 4 bytes and one of 2 bytes,
> meaning there are 4 system calls (select/read/select/read) per 6
> bytes of checksum data).
> One cost of this is some performance, but a significant issue is that
> unbuffered writes generate very short (and very many) ethernet packets,
> which means the overhead is quite large on slow network connections.

... see this link for the rest:

Unsubscribe info:
Bug reporting:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]