[RFC PATCH 0/3] Improved ALSR

[Topi Miettinen]

On 30.11.2020 12.28, Florian Weimer wrote:
> * Topi Miettinen:
> 
>> FYI, the effect of full ASLR of mmap() by kernel also seems small:
>>
>> # echo 3 >/proc/sys/kernel/randomize_va_space
>> # time ./malloc-vs-sbrk
>>
>> real    0m19.489s
>> user    0m1.263s
>> sys     0m18.211s
>> # time ./malloc-vs-sbrk 1
>>
>> real    0m19.532s
>> user    0m1.148s
>> sys     0m18.366s
> 
> The value 3 doesn't do anything in a mainline kernel.

No, you need this patch to randomize mmap(), mremap(..., 
MREMAP_MAYMOVE), stack and vdso:

https://lkml.org/lkml/2020/11/29/214

> I think you will see rather catastrophic effects if you have a workload
> that triggers TLB misses.  I expect that the cost of walking page tables
> increases dramatically with full ASLR.

Each random mapping may likely need new page tables, up to 4 pages. Thus 
for a workload with lots of small (page-size) mappings, the memory 
consumption will indeed increase a lot with 
sysctl.kernel.randomize_va_space=3. But as a real world example, main 
process of Firefox has roughly 1500 lines in /proc/$PID/maps in my 
system. Some are contiguous to other mappings, but for simplicity, 
pretending that they all were independent and each required a full set 
of page tables, it would mean 1500*4*4kB  = 24MB of page tables. RSS of 
the process is 340MB, so the effect is not that dramatic by itself, but 
page table entries may also compete for cache slots. So 
randomize_va_space=3 clearly isn't interesting for performance but 
hardening reasons. I'd also not use it on a 32-bit system because the 
virtual memory space could be fragmented too much. I don't think that 
could happen on a 64 bit system, the physical memory should run out first.

The effect of using mmap() instead of sbrk() in libc is independent of 
the kernel patch: one malloc arena and few other areas will use mmap() 
instead of sbrk(), so only a few mappings are affected. With 
randomize_va_space=2, the mappings made with mmap() will be allocated 
close to the other mappings and then a lot of page tables will be 
shared, so probably no additional memory is needed. With a patched 
kernel and randomize_va_space=3, each mapping will be placed randomly. 
This could take maybe tens of kBs more memory in page tables.

-Topi