RFC: malloc and secure memory.

[Carlos O'Donell]

In reviewing this discussion:
https://github.com/systemd/systemd/pull/14213

The request is for a way to mark some allocations as "secure"
and to give them special properties.

I wonder if we can't do this in some generic way:

- Make arenas a first class construct.

/* Get arena with special properties.  */
malloc_arena *secure_arena = NULL;
/* Get a handle to an arena that has secure heaps.  If glibc can make this
   kind of arena and heap then it does, otherwise it returns NULL.  */
secure_arena = malloc_arena_get (HEAP_SECURE);
/* Does this glibc support his kind of arena?  */
if (secure_arena == NULL)
  abort();

- Bind the malloc call site to a specific arena with specific properties.

For example:

  /* malloc_arena takes an opaque arena pointer that is a global
     variable that the implementation provides, a function pointer
     the memory allocator routine e.g. malloc, and a size.  */
  password_storage = malloc_arena (secure_arena, malloc, size);
  ...
  /* Completely different TU, or scope... */
  free (password_storage);

At the call site you bind:
* The arena.
* The allocator routine.
* The paramters.

At runtime you can:
* Verify if the allocator is one that is part of your own
  implementation, and if it isn't abort (fixes partial
  interposition problem).
* Call the allocator but with the new arena as active
  with the given size.

- Specific arenas have special properties.

For example:
- All allocations from a secure arena use mmap.
- All frees from the secure unmap the memory.

Notes:
- Callers must know the memory may container "secure" information and
  that the eventual user of the memory may place "secure" data in that
  memory. The alternative is significantly more complex in that the actual
  chunk will have to carry the arena type information e.g. pointer to
  an arena or index value. That is to say that the binding of the context
  happens at the *use* site of the memory in question and spreads to the
  entire chunk.

- This concept that an arena should service specific types of memory
  is something that DJ and I have been tossing about for a while.
  Particularly since it might be useful to, at an implementation level
  actually have the properties be specific to the logical heap. For
  example an arena could have a heap that services all large requests.
  So large requests always go via a specific heap in the same arena.
  This creates size-classes which might help alleviate some of the
  workload mixing issues we see with size growth. Likewise the arena
  could have a secure heap that follows these different rules. So
  while the user requests a HEAP_SECURE arena, they will just get
  their local arena but pointing to the secure heap within that arena.

  e.g. 
  struct malloc_arena  {  
    heap_info heap; /* Heap selected by the type.  */
    ... other info ...
  }

- Could be used to specifically request "tagged (coloured) memory"
  like that being offered by aarch64's MTE, and bound to a specific
  allocation only e.g. malloc_arena_get (HEAP_TAGGED); and then
  deal with the consequences of specific chunks that are always
  MTE-enabled.

- I don't want to design a fully pluggable arena interface in C
  with callbacks, just something where we can extend the existing
  interface with a new API.

- Should we consider reducing the heap size to less than 64MiB
  on 64-bit processes?

-- 
Cheers,
Carlos.