Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

[Derry Shribman]

Hi,

 >> Doesn't the 'noacl' mount option provide that already?
 >
 > Partially, there are also the ihash and the exec/notexec options.  A lot
 > has been already discussed on the cygwin-patches list, see, for instance

The problem with mount options is that they are 'static'. They require a cygwin 
'reboot' and they do not allow 'inheritance' for subprocesses, and do not allow 
concurrent processes running in different modes.

Dynamic options via CYGWIN env allow setting stuff in runtime, in /etc/profile, 
~/bashrc, or for specific commands (and their subprocesses), such as:
CYGWIN=no_nlink rsync c:/... z:/...

This allows the user to be free to decide where to relax POSIX compliance in 
order to achieve speed.

It also allows application developers (such as 'git'), to decide in their code 
how they want Cygwin to behave.
In 'git' for example, it does need stat's nlink (number of hard links), and 
actually, not even n_ino (the inode number). Cygwin's git performance was 
ultra-slow, and they improved it by not using Cygwin's stat(), rather 
re-implementing their own 'quick-stat' which worked directly with Win32 API.

If Cygwin would have supported dynamic options (as opposed to mount time 
options), instead of the large 'ifdef __CYGWIN__' code, it would simply be 
adding 'setenv("CYGWIN", "no_nlink no_inode")' to the code in git's main().

This allow applications to declare they will never look into the 'st_ino' and 
'st_nlink'. The authors of an application, at the time of writing it, know 
whether their code accesses these fields or not.

This does not reduce the over-all correctness of the application: cygwin1.dll 
does not collect this extra expensive info - that the application does not use 
anyway.

Many applications can benefit from this, and implementation wise it is quite 
simple, and actually has already been implemented in Yoni's patch!

Derry Shribman

On 9/29/2010 10:32 AM, Corinna Vinschen wrote:
> On Sep 29 05:50, Andy Koppe wrote:
>> On 28 September 2010 18:44, Warren Young wrote:
>>> On 9/28/2010 9:10 AM, Christopher Faylor wrote:
>>>>
>>>> It isn't extremely surprising that Linux access speed for a filesystem
>>>> in a simulated environment, which presumably does not go through
>>>> multiple layers of DLLs, would be faster than Cygwin.
>>>
>>> I think it more likely that the HGFS driver doesn't try to preserve full
>>> POSIX semantics.  There's plenty of precedent: vfat, iso9660...  One could
>>> probably verify this faster by examining the driver's source code
>>> (http://open-vm-tools.sourceforge.net/) than by tracing syscalls.
>>>
>>> If that's the explanation, it points at a possible path forward.
>>>
>>> On Linux, these secondary filesystems aren't expected to provide full POSIX
>>> semantics, simply because they are secondary.  No one cries very hard that
>>> you can't make symlinks on a FAT-formatted USB stick.
>>>
>>> Yet, there's probably no technical reason you couldn't get a POSIX-like
>>> system to run on a crippled filesystem.  It's probably even been done lots
>>> of times before in the embedded world.  Some of the PC Unix systems from the
>>> 80s and early 90s were pretty screwy in this way, too.  Screwy doesn't
>>> prevent you from doing useful work, though.
>>>
>>> Would it not be useful to have a mode in Cygwin that purposely skips any
>>> POSIX semantics that it can't get for free by making the POSIX syscalls
>>> nothing more than thin wrappers around the nearest equivalent Win32 API?  If
>>> you put it in this mode and it breaks, you get to keep both pieces.  There
>>> are those who would happily accept the speed increase for loss of some
>>> functionality.  I wouldn't, but some would.  I'd bet a lot of the 3PPs are
>>> in that group, since they know their target environment very well.
>>
>> Doesn't the 'noacl' mount option provide that already?
>
> Partially, there are also the ihash and the exec/notexec options.  A lot
> has been already discussed on the cygwin-patches list, see, for instance
>
>    http://cygwin.com/ml/cygwin-patches/2010-q3/msg00035.html
>    http://cygwin.com/ml/cygwin-patches/2010-q3/msg00036.html
>
> and
>
>    http://cygwin.com/ml/cygwin-patches/2010-q3/msg00071.html
>    http://cygwin.com/ml/cygwin-patches/2010-q3/msg00073.html
>
>
> Corinna
>