Handwaving RFC: hypothetical recalloc() API

[H. Peter Anvin]

One thing that has bothered me for a very long time when implementing user
space code is the realloc() + memset() anti-pattern.

It is an antipattern for multiple reasons:

1. memory allocated from the operating system is already clear, and explicitly
zeroing it userspace causes it to be COW'd even when unnecessary -- which it
might very well be as it is more and more becoming standard practice to always
zero out a buffer on allocation -- for good reasons.

2. If the memory was allocated by calloc() in the first place, then it is
likely zeroed to its entire buffer size, not just the part that was requested.

3. realloc() lacks the often-vaunted multiplication overflow protection of
calloc(), rendering the latter useless if realloc() is used on the buffer
anywhere. (recallocarray() does have this feature.)

4. Neither provides any indication of the buffer size *actually* allocated, as
opposed to requested.

5. There is no realloc() equivalent of aligned_alloc/memalign.


I'm thinking something like:


int recalloc(void * restrict * restrict old, size_t alignment,
	     size_t * restrict n, size_t size);


On success, *old is deallocated unless NULL, and set to the new buffer, which
may or may not be the same as the old buffer. *n will be set to the number of
<size> items actually allocated, which will be greater than or equal to the
incoming value, and the function returns zero.

All newly allocated memory, if any, is cleared. There is a potential hazard
here: unless the existing malloc(), realloc() etc. functions are modified to
always clear any spillover area (the difference between the buffer requested
and the buffer allocated) then this would mean that the buffer size originally
requested must be stored in the hidden buffer metadata. That clearing,
however, might be desirable anyway for safety reasons (see above.)

If *n == 0, the function returns a unique pointer value (let's not do the
"unique or NULL" garbage that is literally the worst of both worlds) that can
be passed to realloc(), recalloc() or free() but is not in any way guaranteed
to be otherwise usable, nor aligned to <alignment> [the latter point is
questionable; however, if <alignment> is very large this may cause an
unacceptable. I am not sure how posix_memalign() deals with this case.]

On failure, *old is not deallocated (and is thus unmodified), *n is set to the
actual size of the existing buffer (which may be equal to larger than the
input *n in case the input *old did not meet the desired alignment) and the
function returns an errno value.

I *really* dislike the posix_memalign() requirement that the alignment needs
to be a multiple of sizeof(void *). It makes absolutely no sense; it is
perfectly reasonable to specify any alignment down to single bytes,
*especially* since even standard C now has an alignof() operator. For the
allocator to not accept a value returned by alignof() is very serious brain
damage. Furthermore, it is completely pointless: the allocator is obviously
always permitted to return a larger alignment than requested.

I would also suggest that an alignment of 0 would explicitly be defined as
"the alignment of the largest type that can fit inside *n * size". Including
*n in the calculation is because of the common-enough pattern with calloc() to
always provide a byte count, regardless of the intended use of the buffer.

A nice addition would be an explicit macro similar to:

#define recalloc_for(ptr_p,n_p) \
	recalloc((void **)(ptr_p),_Alignof(*(ptr_p)),(n_p),sizeof(*(ptr_p)))

[adding appropriate compiler magic to make the ptr_p cast saner.]

	-hpa