Memory layout patch

[Igor Podgainoi]

Thank you for your reply!

On Jan 20 12:58, Corinna Vinschen wrote:
> Question here is: Is just ASLR enforced or High Entropy ASLR?
>
> If the first, we have to make sure to compile all Cygwin binaries as
> non-HEVA.
>
> If the latter, we have a really hard fork() problem which can't be
> fixed by your patch...

According to my latest tests, only ASLR itself is enforced
(IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE), while High Entropy ASLR
(IMAGE_DLL_CHARACTERISTICS_HIGH_ENTROPY_VA) can successfully be
turned off at least on Windows 11 23H2 build 10.0.22631.5909.

Also, it looks like there are no more memory collision problems
after disabling HEVA. The tests ltp/fork06.exe, ltp/fork07.exe and
ltp/fork11.exe pass successfully even with an increased number of
iterations. Other tests also do not seem to be affected.

The location of the large FREE block seems to be more predictable too.
Here are a few examples I encountered across reruns:
+        0`7ffe1000     7ff4`b7e50000     7ff4`37e6f000             MEM_FREE    PAGE_NOACCESS                      Free

+        0`7ffe1000     7ff4`fb740000     7ff4`7b75f000             MEM_FREE    PAGE_NOACCESS                      Free

+        0`7ffe1000     7ff4`f3cd0000     7ff4`73cef000             MEM_FREE    PAGE_NOACCESS                      Free

+        0`7ffe1000     7ff4`14010000     7ff3`9402f000             MEM_FREE    PAGE_NOACCESS                      Free

+        0`7ffe1000     7ff4`b5ca0000     7ff4`35cbf000             MEM_FREE    PAGE_NOACCESS                      Free

This means that modifying the Makefile.am files to fix this issue
should be enough. I am planning to post a patch to the Cygwin-patches
mailing list disabling HEVA by adding -Wl,--disable-high-entropy-va
where necessary. This would allow the three unit tests I had mentioned
to pass.

What are your thoughts on this? In my opinion, it should be fine at
least as a temporary solution, until the binutils/GCC and Cygwin
communities figure out how to properly fix HEVA support in Cygwin
on AArch64.

> Leaving HEVA aside, the usual memory usage of the OS on x86_64 is so
> that the OS is basically using the lower 2 Gigs (stacks, heaps) and the
> upper N Gigs for PEB, TEBs and DLLs.  That leaves the huge middle of the
> VM free for the application.
>
> The memory layout of Cygwin tries to accomodate that by starting at the
> second 4 Gigs block with the executable image (EXECUTABLE_ADDRESS), and
> ending at a reasonable high address with memory maps allocated top-down
> (MMAP_STORAGE_HIGH).
>
> If you're observing PEB and TEB reservations in the area from
> 0x800000000 up to 0xa00000000 without HEVA, then that's IMHO a design
> flaw in the ARM64 memory allocator, which should be fixed by Microsoft.

Thankfully, that seems not to be the case with HEVA disabled - I had
not thought to disable it prior to you mentioning it in your email.

> > +#define CYGHEAP_STORAGE_LOW                0x0ff700000000UL
> > +#define CYGHEAP_STORAGE_INITIAL    0x0ff700300000UL
> > +#define CYGHEAP_STORAGE_HIGH               0x0ff900000000UL
> > [...]
> > +#define USERHEAP_START             0x0ff900000000UL
> >  [...]
> > +#define MMAP_STORAGE_LOW           0x0fff00000000UL
> > +#define MMAP_STORAGE_HIGH          0x7fef00000000UL
>
> I'm a bit puzzeled that you're moving MMAP_STORAGE_HIGH up,
> from 0x700000000000UL.  Granted, this may be somewhat over the top,
> but the general idea was to allow a lot of space left for ASLR'ed
> DLLs.  Other than that, I don't see much of a problem.  Your patch
> restricts the VM usable for heap and maps to a mere 112 TB, which
> is, of course, worrisome /*grin*/

My thought process behind this was to shift everything up uniformly -
I was not sure where userheap ends and whether there is a gap between
userheap and mmap_storage.

> > * or whether there is a more appropriate approach to handling this
> >   under mandatory ASLR.
>
> It all depends on the fact if HEVA is mandatory, or just ASLR with
> default characteristics.  For normal ASLR we can redefine the memory
> layout as we see fit.  The DLL areas REBASED_DLL_STORAGE_LOW/
> REBASED_DLL_STORAGE_HIGH and AUTOBASED_DLL_STORAGE_LOW/
> AUTOBASED_DLL_STORAGE_HIGH will be basically unused because all DLLs
> will be ASLR'ed anyway.  IMHO more reason to stick to the current
> MMAP_STORAGE_HIGH, but that's a point of discussion.

That is useful to keep in mind, thanks.

> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential [etc]
>
> Tell your employer to drop these disclaimers for postings to public
> mailing lists.  They just don't make any sense.

Should be fixed now. Hopefully this email does not include it anymore,
fingers crossed.

Best regards,
Igor Podgainoi
Arm