Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

[Eric Blake]

[please quit top-posting]

On 09/29/2010 12:29 PM, Derry Shribman wrote:
> Hi,
>
> xstat() is a new Linux system call that allows a user-mode application
> to specifically request more (or less) stat information.

Is it even in the Linux kernel yet?  The last I found on my quick 
googling is that it is still under discussion.

>
> Since this is a new Linux API, it requires porting for existing Unix
> application to replace every single stat() calls in their code to
> xstat() in order to benefit from this API.

Yes, but several projects, like coreutils, are already committed to this 
task, and since coreutils covers the most common apps you use on a daily 
basis, it will have immediate effect.

>
> This is quite a big change in the code - and application maintainers
> will not rush to change their code until this new Linux kernels with
> this API are commonly installed.

Ah, but you are forgetting one thing.  xstat() can be written as a 
library wrapper around stat() on systems that lack the xstat() kernel 
call.  True, the library wrapper won't be as fast, but at least that 
way, applications can be written that blindly use xstat() even before 
kernel support is widespread.  The _exact same thing_ happened with 
coreutils' early adoption of openat() and friends for faster directory 
traversal - coreutils was using the gnulib openat() wrapper prior to 
POSIX 2008 standardizing openat() (in particular, the cygwin port of 
coreutils was blindly calling openat() and working prior to cygwin 
implementing openat()!).

Therefore, once the xstat() kernel interface is nailed down, the 
coreutils/gnulib developers WILL start using it in preference to stat(), 
and as soon as cygwin supplies xstat(), cygwin will immediately see the 
benefits.

> So it does not fit what Cygwin needs because:
> 1) it requires a lot of change to existing code of every single
> application.

Yes, but so does using openat() for improved robustness and speed.

> 2) it will take many years until it will be available on most of the
> installed Linux kernel images.

For porting purposes, it is irrelevant whether it is available in the 
kernel or a library (obviously, the speed factor won't happen unless it 
is in the kernel, but that will come).

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org