This is the mail archive of the cygwin mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Broken autoconf mmap test (was Re: 1.7] BUG - GREP slows to a crawl with large number of matches on a single file)

Hash: SHA1

According to Corinna Vinschen on 11/9/2009 7:05 AM:
> This part of the testcase
>   data2 = (char *) malloc (2 * pagesize);
>   if (!data2)
>     return 1;
>   data2 += (pagesize - ((long int) data2 & (pagesize - 1))) & (pagesize - 1);
>   if (data2 != mmap (data2, pagesize, PROT_READ | PROT_WRITE,
>                        MAP_PRIVATE | MAP_FIXED, fd, 0L))
>     return 1;
> is bad.  The chance that the address of data2 is not usable for mmap on
> Windows/Cygwin is 100%.

But in testing this further, I discovered that you CAN do:

data2 = mmap(...);
munmap (data2,...);
mmap (data2, ... MAP_FIXED)

and get success on cygwin.  So I will be updating autoconf accordingly,
based on the STD below.  Unfortunately, it looks like I also found a hole
in cygwin.  Consider this (borrowing heavily from the autoconf test that I
am fixing):

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>

main (int argc, char **argv)
  char *data, *data2, *data3;
  int i, pagesize;
  int fd, fd2;

  pagesize = getpagesize ();
  /* First, make a file with some known garbage in it. */
  data = (char *) malloc (pagesize);
  if (!data)
    return 1;
  for (i = 0; i < pagesize; ++i)
    *(data + i) = rand ();
  umask (0);
  fd = creat ("conftest.mmap", 0600);
  if (fd < 0)
    return 2;
  if (write (fd, data, pagesize) != pagesize)
    return 3;
  close (fd);

  /* Next, check that a page is zero-filled if not backed by a file.  */
  fd2 = open ("conftest.txt", O_RDWR | O_CREAT | O_TRUNC, 0600);
  if (fd2 < 0)
    return 11;
  data2 = "";
  if (write (fd2, data2, 1) != 1)
    return 12;
    /* We expect mmap to succeed, but reads to give SIGBUS, since mapped
       region is an entire page beyond bounds of mapped file.  */
  data2 = mmap (0, pagesize, PROT_READ | PROT_WRITE, MAP_SHARED, fd2, 0L);
  if (data2 == MAP_FAILED)
    return 14;
  printf ("mapped %p\n", data2);
  for (i = 0; i < pagesize; ++i)
    if (*(data2 + i))
        printf ("%p, %x\n", data2 + i, *(data2 + i));
        return 15;
  close (fd2);
  if (argc > 1)
    munmap (data2, pagesize);

  /* Next, try to mmap the file at a fixed address which already has
     something else allocated at it.  If we can, also make sure that
     we see the same garbage.  */
  fd = open ("conftest.mmap", O_RDWR);
  if (fd < 0)
    return 4;
  if (data2 != mmap (data2, pagesize, PROT_READ | PROT_WRITE,
                   MAP_PRIVATE | MAP_FIXED, fd, 0L))
    return 6;
  for (i = 0; i < pagesize; ++i)
    if (*(data + i) != *(data2 + i))
        printf ("%p, exp %x, got %x\n", data2 + i, *(data + i), *(data2 + i));
        return 7;

  /* Finally, make sure that changes to the mapped area do not
     percolate back to the file as seen by read().  (This is a bug on
     some variants of i386 svr4.0.)  */
  for (i = 0; i < pagesize; ++i)
    *(data2 + i) = *(data2 + i) + 1;
  data3 = (char *) malloc (pagesize);
  if (!data3)
    return 8;
  if (read (fd, data3, pagesize) != pagesize)
    return 9;
  for (i = 0; i < pagesize; ++i)
    if (*(data + i) != *(data3 + i))
      return 10;
  close (fd);
  return 0;

This test behaves differently on Linux than on cygwin; on Linux, both
'./foo' and './foo 1' give status 0, but on cygwin, './foo' gives status
6, and only './foo 1' succeeds.  In other words, the second mmap fails if
there is no intermediate munmap.

POSIX apparently allows cygwin's behavior:

"If MAP_FIXED is set, mmap() may return MAP_FAILED and set errno to
[EINVAL]. If a MAP_FIXED request is successful, the mapping established by
mmap() replaces any previous mappings for the pages in the range
[pa,pa+len) of the process."

However, since we already have to maintain a list of mappings in order to
implement fork(), it seems like it would be easy to fix cygwin to
implicitly munmap anything that would otherwise be in the way of a
subsequent MAP_FIXED request, rather than blindly calling
NtMapViewOfSection and failing because of the overlap, so that we could be
even more like Linux behavior.

> That's why I think we need at least two tests in autoconf, a generic
> mmap test and a mmap test for the "mmap private/shared fixed at
> somewhere already mapped" case, if an application actually insists on
> using that.

In the case of the autoconf test, I think a single test is still
sufficient, once it is fixed to be portable to what POSIX requires.

gnulib provides a more interesting test, for whether MMAP_ANON works.

- --
Don't work too hard, make some time for fun as well!

Eric Blake   
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at
Comment: Using GnuPG with Mozilla -


Problem reports:
Unsubscribe info:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]