Atomic mmap replacement

[Ken Brown]

On 12/14/2024 7:00 PM, Ken Brown via Cygwin wrote:
> Hi Corinna,
> 
> On 2/19/2018 4:00 AM, Corinna Vinschen wrote:
>> On Feb 17 22:37, Ken Brown wrote:
>>> Some code in emacs wants to reserve a chunk of address space with a big
>>> PROT_NONE anonymous mapping, and then carve it up into separate mappings
>>> associated to segments of a file.  This fails on Cygwin.
> 
> [...]
> 
>> Several limitations in the Windows kernel disallow this:
> 
> [...]
> 
>> - It also disallows to re-map any allocated or mapped mamory for another
>>    purpose.
>>
>> So this part of the POSIX specs for mmap:
>>
>>    "The mapping established by mmap() shall replace any previous mappings
>>     for those whole pages containing any part of the address space of the
>>     process starting at pa and continuing for len bytes"
>>
>> can't be implemented with Windows means.
> 
> I'm returning to this very old thread because of come up against another 
> application that wants to allocate a big block of memory and then 
> allocate pieces of it later.  I've looked at the documentation of 
> VirtualAlloc, and it seems that this should be possible:
> 
>     VirtualAlloc cannot reserve a reserved page. It can commit a page
>     that is already committed. This means you can commit a range of
>     pages, regardless of whether they have already been committed, and
>     the function will not fail.
> 
>     You can use VirtualAlloc to reserve a block of pages and then make
>     additional calls to VirtualAlloc to commit individual pages from
>     the reserved block. This enables a process to reserve a range of
>     its virtual address space without consuming physical storage until
>     it is needed.
> 
> The attached test case illustrates this.  Do you think it's feasible to 
> modify mmap to take advantage of this?  If you try to mmap a block 
> that's inside an already allocated block, I would want mmap to call 
> VirtualAlloc with MEM_COMMIT but not MEM_RESERVE, and this should succeed.
> 
> If you think this is feasible, I would be willing to work on it.  But in 
> that case I would appreciate some suggestions on how to implement it, 
> since I'm not yet very familiar with the mmap code.
It looks like a lot of the machinery for doing what I want is already 
present in mmap.cc.  If I want the initial allocation to reserve without 
committing [in the Windows sense of "reserve"], I just need to specify 
MAP_NORESERVE in the call to mmap [now we're using "noreserve" in the 
Linux sense].  Right?  Then future mmap calls to allocate memory within 
that first block could simply check for the noreserve flag and use 
MEM_COMMIT without MEM_RESERVE.  Obviously there are a lot of details 
that I haven't yet thought through, but I'm cautiously optimistic.

Ken

P.S. The conflicting meaning of "reserve" in Windows vs. Linux was very 
confusing to me at first.  There's probably nothing that can be done to 
make the code less confusing.