Performance optimization in av::fixup - use buffered IO, not mapped file

[Ryan Johnson]

On 12/12/2012 10:39 AM, Ryan Johnson wrote:
> On 12/12/2012 9:04 AM, Eric Blake wrote:
>> On 12/12/2012 06:22 AM, Corinna Vinschen wrote:
>>> On Dec 12 06:11, Eric Blake wrote:
>>>> On 12/11/2012 08:13 PM, Daniel Colascione wrote:
>>>>> Considering the horrible and
>>>>> unexpected performance implications of sparse files, I don't think 
>>>>> generating
>>>>> them automatically from a sequence of seeks and writes is the 
>>>>> right thing to do.
>>>> Why can't we instead use posix_fallocate() as a means of identifying a
>>>> file that must not be sparse, and then just patch the compiler to use
>>>> posix_fallocate() to never generate a sparse executable (but let all
>>>> other sparse files continue to behave as normal)?
>>>>
>
>>> posix_fallocate is not allowed to generate sparse files, due to the 
>>> following restriction: "If posix_fallocate() returns successfully, 
>>> subsequent writes to the specified file data shall not fail due to 
>>> the lack of free space on the file system storage media." See 
>>> http://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_fallocate.html 
>>> Therefore only ftruncate and lseek potentially generate sparse 
>>> files. On second thought, I don't quite understand what you mean by 
>>> "use posix_fallocate() as a means of identifying a file that must 
>>> not be sparse". Can you explain, please? 
>> Since we know that an executable must NOT be sparse in order to make it
>> more efficient with the Windows loader, then gcc should use
>> posix_fallocate() to guarantee that the file is NOT sparse, even if it
>> happens to issue a sequence of lseek() that would default to making it
>> sparse without the fallocate.
>>
>> In other words, I'm proposing that we delete nothing from cygwin1.dll,
>> and instead fix the problem apps (gcc, emacs unexec) that actually
>> create executables, so that the files they create are non-sparse because
>> we have proven that they should not be sparse for performance reasons.
>> Meanwhile, all non-executable files (such as virtual machine disk
>> images, which are typically much bigger than executables, and where
>> being sparse really does matter) do not have to jump through extra hoops
>> of using ftruncate() when plain lseek() would do to keep them sparse.
> Does gcc/ld/whatever know the final file size before the first write?
>
> You have to posix_fallocate the entire file before any write that 
> might create a hole, because the sparse flag poisons the loader, and 
> persists even if all gaps are later filled. 
Heh... hit send too soon.

Alternatively, a quick experiment verifies that calling pwrite instead 
of lseek+write bypasses the sparse-ifying "optimization." If emacs and 
binutils always seek before writing, it might be as simple as patching 
them to use pwrite instead. That would even improve performance on other 
platforms with pwrite, by cutting the syscall count in half.

A quick scan of binutils sources suggests that all*** file writes go 
through libiberty/simple-object.c:simple_object_internal_write, which 
indeed uses an lseek+write pair. As long as gcc uses libiberty to write 
out executables as well, it should pick up the fix automatically.

The emacs (24.0.96) unexecw.c copies the entire executable file over 
using read/write pairs (= not sparse), and then patches the output using 
seek/write pairs. Again, an easy conversion to use pread instead.

*** I'm actually not sure what gold does, but we don't care because it 
doesn't target cygwin anyway.

Ryan