Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

[Derry Shribman]

Hi,

 > Yes, but several projects, like coreutils, are already committed to this task,
 > and since coreutils covers the most common apps you use on a daily basis, it
 > will have immediate effect.

from 'committed' to actually coding the code, and then releasing it, and then 
'cygwin ports' project updating to use it: this will takes years, and this is 
only core-utils. When will cvs, git, gnumake, apache etc use it...?

Also, what about a developer with a large (slow) perl application? he will have 
to wait for perl to use and export xstat API, and then until his application 
will specifically replace stat -> xstat, and only then the application will work 
faster. This can take years.

Derry

On 9/29/2010 9:41 PM, Eric Blake wrote:
> [please quit top-posting]
>
> On 09/29/2010 12:29 PM, Derry Shribman wrote:
>> Hi,
>>
>> xstat() is a new Linux system call that allows a user-mode application
>> to specifically request more (or less) stat information.
>
> Is it even in the Linux kernel yet? The last I found on my quick googling is
> that it is still under discussion.
>
>>
>> Since this is a new Linux API, it requires porting for existing Unix
>> application to replace every single stat() calls in their code to
>> xstat() in order to benefit from this API.
>
> Yes, but several projects, like coreutils, are already committed to this task,
> and since coreutils covers the most common apps you use on a daily basis, it
> will have immediate effect.
>
>>
>> This is quite a big change in the code - and application maintainers
>> will not rush to change their code until this new Linux kernels with
>> this API are commonly installed.
>
> Ah, but you are forgetting one thing. xstat() can be written as a library
> wrapper around stat() on systems that lack the xstat() kernel call. True, the
> library wrapper won't be as fast, but at least that way, applications can be
> written that blindly use xstat() even before kernel support is widespread. The
> _exact same thing_ happened with coreutils' early adoption of openat() and
> friends for faster directory traversal - coreutils was using the gnulib openat()
> wrapper prior to POSIX 2008 standardizing openat() (in particular, the cygwin
> port of coreutils was blindly calling openat() and working prior to cygwin
> implementing openat()!).
>
> Therefore, once the xstat() kernel interface is nailed down, the
> coreutils/gnulib developers WILL start using it in preference to stat(), and as
> soon as cygwin supplies xstat(), cygwin will immediately see the benefits.
>
>> So it does not fit what Cygwin needs because:
>> 1) it requires a lot of change to existing code of every single
>> application.
>
> Yes, but so does using openat() for improved robustness and speed.
>
>> 2) it will take many years until it will be available on most of the
>> installed Linux kernel images.
>
> For porting purposes, it is irrelevant whether it is available in the kernel or
> a library (obviously, the speed factor won't happen unless it is in the kernel,
> but that will come).
>