Fix performance on 10Gb networks
Iuliu Rus
rus.iuliu@gmail.com
Wed Nov 19 18:18:00 GMT 2014
You are right, of course. We initially thought it has to be a
multiple of page_size but it doesn't. I just re-tested with 63k and it
gives good perf too.
We get 600Mbits/second compared with 10Mb for the old default.
Attached the modified patch.
On Tue, Nov 18, 2014 at 8:43 PM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> Hi Iuliu,
>
> On Nov 18 19:30, Iuliu Rus wrote:
>> Hello,
>> Google is running Cygwin apps on its 10Gb networks and we are seeing
>> extremely bad performance in a couple of cases. For example, iperf
>> with the defaults results in only 10Mbits/sec.
>> We tracked this down to a combination of non-blocking sockets with
>> Nagle+delayed ack kicking in, since the apps eventually end up sending
>> a very small packets (2 bytes).
>> We have a case open against Microsoft but since everything is moving
>> very slow we would like to work around by picking socket buffers that
>> are multiple of 4k.
>
> Thanks for the patch. One question:
>
>> Change log:
>> 2014-11-18 Iuliu Rus <rus.iuliu@gmail.com>
>>
>> * net.cc Change default values for socket buffers to fix performance
>> on 10Gb networks.
>>
>> Index: winsup/cygwin/net.cc
>> ===================================================================
>> RCS file: /cvs/src/src/winsup/cygwin/net.cc,v
>> retrieving revision 1.320
>> diff -u -p -r1.320 net.cc
>> --- winsup/cygwin/net.cc 13 Oct 2014 08:18:18 -0000 1.320
>> +++ winsup/cygwin/net.cc 18 Nov 2014 19:12:00 -0000
>> @@ -621,13 +621,16 @@ fdsock (cygheap_fdmanip& fd, const devic
>> this is no problem on 64 bit. So we set the default buffer size to
>> the default values in current 3.x Linux versions.
>>
>> - (*) Maximum normal TCP window size. Coincidence? */
>> + (*) Maximum normal TCP window size. Coincidence?
>> +
>> + NOTE 3. Setting the window size to 65535 results in extremely
>> bad performance for apps that send data in multiples of Kb, as they
>> eventually end up sending 1 byte on the network and naggle + delay ack
>> kicks in. For example, iperf on a 10Gb network gives only 10 Mbits/sec
>> with a 65535 send buffer. We want this to be a multiple of PAGE_SIZE,
>> but since 64k breaks WSADuplicateSocket we use 60Kb.
>
> We do? See below.
>
>> +*/
>> #ifdef __x86_64__
>> ((fhandler_socket *) fd)->rmem () = 212992;
>> ((fhandler_socket *) fd)->wmem () = 212992;
>> #else
>> - ((fhandler_socket *) fd)->rmem () = 65535;
>> - ((fhandler_socket *) fd)->wmem () = 65535;
>> + ((fhandler_socket *) fd)->rmem () = 63488;
>> + ((fhandler_socket *) fd)->wmem () = 63488;
>
> This is 62K, certainly not a multiple of the native PAGE_SIZE of 4K.
> And this makes me wonder. Did you intend to use 60K and ended up with
> 62K for a reason? And then, why not 63K as a multiple of 1K?
>
>
> Corinna
>
> --
> Corinna Vinschen Please, send mails regarding Cygwin to
> Cygwin Maintainer cygwin AT cygwin DOT com
> Red Hat
-------------- next part --------------
A non-text attachment was scrubbed...
Name: net_patch
Type: application/octet-stream
Size: 1445 bytes
Desc: not available
URL: <http://cygwin.com/pipermail/cygwin-patches/attachments/20141119/01e88f89/attachment.obj>
More information about the Cygwin-patches
mailing list