Performance optimization in av::fixup - use buffered IO, not mapped file

Daniel Colascione
Wed Dec 12 21:21:00 GMT 2012

On 12/12/2012 12:09 PM, Ryan Johnson wrote:
> On 12/12/2012 2:21 PM, Christopher Faylor wrote:
>> On Wed, Dec 12, 2012 at 12:11:46PM -0500, Ryan Johnson wrote:
>>> On 12/12/2012 12:03 PM, Christopher Faylor wrote:
>>>> On Tue, Dec 11, 2012 at 07:13:04PM -0800, Daniel Colascione wrote:
>>>>> On 12/11/2012 5:06 PM, Daniel Colascione wrote:
>>>>>> On 12/10/2012 7:51 PM, Daniel Colascione wrote:
>>>>>>> The key to generating a binary that repros the problem is to unexec
>>>>>>> emacs, then
>>>>>>> try to repro with that generated binary, not a copy of it.
>>>>>> The real explanation is a lot simpler: the binary is sparse. When you
>>>>>> create a
>>>>>> file mapping object for a sparse file, Windows discards all cached pages for
>>>>>> that file. It makes sense that compilers (and Emacs unexec) would create
>>>>>> sparse
>>>>>> files as they seek around inside their outputs.
>>>>> Anyway, the binary is sparse because our linker produces sparse files.
>>>>> Would the Cygwin developers accept this patch? With it, applications would
>>>>> need
>>>>> to explicitly use ftruncate to make files sparse. Considering the horrible and
>>>>> unexpected performance implications of sparse files, I don't think generating
>>>>> them automatically from a sequence of seeks and writes is the right thing
>>>>> to do.
>>>> I don't know if this was already done (don't see it in a quick glance at
>>>> the archives) but, if this is just a simple case of executable files
>>>> being sparse, it seems like an obvious optimization would be to just to
>>>> do a, e.g.,
>>>> cp --sparse=never -p foo.exe foo.exe.tmp
>>>> mv foo.exe.tmp foo.exe
>>>> Wouldn't that remove the sparseness and wouldn't you see astounding
>>>> performance improvments as a result?
>>> Nope. You'd have to rm foo.exe first.
>> "mv" does that automatically, doesn't it?
> Oops. Thinko. I had "cp" in my head.
>>> Doing so fixes the problem nicely, though, as you suggest.
>>>> I don't think we should be considering ripping code out of Cygwin
>>>> without some actual data to back up claims.  Testing something like the
>>>> above should make it easier to justify.
>>>> I'm actually rather surprised that setup.exe's tar code would maintain an
>>>> executable's sparseness.
>>> Setup is fine. It's home-brew stuff that suffers, unless/until invoking
>>> `make install' copies the sparse file to its final destination, losing
>>> the sparse property along the way.
>>> Personally, I'm still in shock that the loader barfs so badly over
>>> sparse files... normal reads via mmap and fread use the fs cache just fine.
>> If we're talking about the loader then why are the examples I asked for
>> using "cp"?  That's not really apples-to-apples.
> I think I'm miscommunicating something here... let me try again.
> Problem:
> Attempts to execute a file with the NTFS "sparse" attribute force the Windows
> loader to bypass the fs cache and fetch the file from disk.

Almost. What actually happens is that the sparse attribute forces the system to
flush the cache _for the whole file_ when av::fixup mmaps the binary, forcing
the subsequent execution of the image we mmapped to reload everything from disk.
Cygexec will work around the problem by avoiding the av::fixup mmap, but we
shouldn't expect users to mount everything, including various build directories,
cygexec. (I also haven't checked whether anything in /bin actually ends up
sparse. It'd be interesting to see.)

We could also work around the problem by making av::fixup use regular buffered
reads instead of mmap. Regular buffered reads don't seem to trigger the cache flush.

Still, I think that creating sparse files only on ftruncate is the right thing
to do, at least for existing OS versions. Basically, almost nobody _uses_ sparse
files in Windows (except us), so the paths that deal with them are optimized for
simplicity and correctness, not performance.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
URL: <>

More information about the Cygwin-developers mailing list