Broken autoconf mmap test

Corinna Vinschen corinna-cygwin@cygwin.com
Thu Mar 24 10:00:00 GMT 2011


On Mar 23 19:15, Corinna Vinschen wrote:
> On Mar 23 11:51, Eric Blake wrote:
> > I'm still debating if cygwin is compliant and the autoconf test is
> > exploiting undefined behavior... this is tricky stuff to get right.
> 
> Cygwin tries to be as compliant as possible, and it tries to implement
> expectations which are typical for some Unix systems even if they are
> not covered by the standards.
> 
> And yes, the autoconf test tests defined behaviour, see the SUSv4 mmap
> man page:
> [etc]

For the records and, hopefully, even for discussion if somebody feels
a bit of pity with me.

I've been mulling over this problem on and off for a long time.  I just
don't see a solution for 64 bit systems by just using the OS calls as
they are defined.  AT_ROUND_TO_PAGE is not supported, AT_EXTENDABLE_FILE
changes the file size.

So I was thinking of an entirely different way to implement mmap's on
real files: Instead of really mapping them, mmap just generates an
anonymous map.

In case of MAP_PRIVATE, anonymous maps are implemented using
VirtualAlloc.  The size of the allocated space is always a multiple of
64K.  After creating it, the file content is simply copied over.
Full stop.  Easy.

In case of MAP_SHARED, anonymous maps are created using an actual
file mapping (NtCreateSection/NtMapViewOfSection).  For files, mmap
would still simply copy over the file content.

The problem starts if the map has been opened using PROT_WRITE.  In that
case we have to write the changed pages back to the file.  Fortunately
the only times you can rely on the changes being written back to the
file are msync, munmap, and process exit.

But that's the tricky part.  How do we know a page has been written
to?

Easy solution:  Always write all of the mapping to the file.  Would that
be possible?  This sounds wrong to me.  IIUC, msync should only write
back the changes, not the entire content of the mapping.  It would
potentially overwrite changes made in the file by other means.

Starting with Windows 2000 there's a method to keep track of pages which
have been written to:  Call VirtualAlloc with the MEM_WRITE_WATCH flag
set, and then use GetWriteWatch to find the changed pages.
Unfortunately the MEM_WRITE_WATCH flag works only for VirtualAlloc'ated
memory, not for section maps created with MapViewOfFile, so it's not
usable in this scenario.  We can't use VirtualAlloc for the MAP_SHARED
case, otherwise the changes in memory are not visible to other processes
sharing the map.

What's left? 

The hard way, apparently.  Always set the page protection to
PAGE_READONLY.  If the process writes to it, an exception occurs.  Catch
the exception, set the page protection to PAGE_READWRITE and return from
the exception so the caller can write again.  When the changes are to be
written back to the file, only write back the pages with PAGE_READWRITE
protection.  If this is called from msync, reset the page protection to
PAGE_READWRITE.

Apart from being complicated and error prone, there are a couple of
problems I can see:

- The protection set by mmap and mprotect doesn't match the actual
  protection.  So we need to keep track of the desired protection
  per page.  This takes one byte per page on the cygheap.  So far we
  only needed 1 bit per page.

- It's darn tricky to make this thread-safe.

- In all three possible solutions above: What if the original file
  handle used in the mmap call has been closed and the file permissions
  have been changed in the meantime so that the process does not have
  write permissions anymore?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat



More information about the Cygwin-developers mailing list