glibc realloc(ptr, 0) vs. C17 specification

[Eric Blake <eblake at redhat dot com>]

tl;dr: the POSIX folks need glibc developer input on what appears to be
a contradiction between the C17 standard and glibc realloc(ptr,0) behavior.

Full details:
The Austin Group (the organization in charge of the POSIX specification)
is working towards the next version of POSIX; the current POSIX 2017
(Issue 7) is based on C99, but the next version of POSIX (several years
out still) will be based on C17.  But in preparing to update the C
standard used by POSIX, we ran into a change in the C standard that
appears on the surface to either be at odds with the glibc behavior, or
to be internally self-inconsistent.

First, some background:

In 2011, I opened http://austingroupbugs.net/view.php?id=374, which
tried to get clarification about the behavior of malloc(0) and
realloc(p, 0), in light of the fact that some implementations treat an
attempt to allocate 0 bytes as an error, while others (like glibc) let
malloc(0) behave the same as malloc(1) except that you should not
dereference the pointer, only pass it to free().

In the course of discussing that, the Austin Group realized that there
were multiple libc realloc behaviors in the wild, all of which seemed to
be compliant with various loopholes in C11:

AIX: realloc(NULL,0) always returns NULL, errno is EINVAL
      realloc(ptr,0) always returns NULL, ptr freed, errno is EINVAL
BSD: realloc(NULL,0) only gives NULL on alloc failure, errno is ENOMEM
      realloc(ptr,0) only gives NULL on alloc failure, ptr unchanged,
        errno is ENOMEM
glibc:realloc(NULL,0) only gives NULL on alloc failure, errno is ENOMEM
      realloc(ptr,0) always returns NULL, ptr freed, errno unchanged

The C11 wording was:

C11 7.22.3 (overview): If the size of the space requested is zero, the
behavior is implementation-defined: either a null pointer is returned,
or the behavior is as if the size were some nonzero value, except that
the returned pointer shall not be used to access an object.

C11 7.22.3.5 (realloc): Otherwise, if ptr does not match a pointer
earlier returned by a memory management function, or if the space has
been deallocated by a call to the free or realloc function, the behavior
is undefined. If memory for the new object cannot be allocated, the old
object is not deallocated and its value is unchanged.
The realloc function returns a pointer to the new object (which may have
the same value as a pointer to the old object), or a null pointer if the
new object could not be allocated.

But because the compliance relied on loopholes in C11, a C defect was
raised:

http://open-std.org/JTC1/SC22/WG14/www/docs/dr_400.htm

and C17, as a result, has changed the requirements as follows:

C17 7.22.3.1 (overview): If the size of the space requested is zero, the
behavior is implementation-defined: either a null pointer is returned to
indicate an error, or the behavior is as if the size were some nonzero
value, except that the returned pointer shall not be used to access an
object.

C17 7.22.3.5 (realloc): If size is non-zero and memory for the new
object is not allocated, the old object is not deallocated. If size is
zero and memory for the new object is not allocated, it is
implementation-defined whether the old object is deallocated.
The realloc function returns a pointer to the new object (which may have
the same value as a pointer to the old object), or a null pointer if the
new object has not been allocated.

C17 7.31.12 Invoking realloc with a size argument equal to zero is an
obsolescent feature.

By a strict reading of 7.22.3.1 in isolation, this changes the
requirements to state that realloc(a, b) cannot return NULL except to
indicate an error (whereas C11 allowed a return of NULL even on
success); but this is at odds with glibc behavior (where realloc(ptr,0)
frees ptr but returns NULL as an indication of success).

However, a reading of 7.22.3.5 makes it appear that a NULL return when
nothing is allocated (which is the case for realloc(ptr, 0)) is
acceptable, even though it does not indicate an error, and therefore
disagrees with the earlier statement that a null pointer indicates an
error.  And the fact that the wording still leaves it
implementation-defined whether the old pointer was freed or left
allocated gets back to the conundrum that POSIX is trying to solve, by
specifying either specific errno values (preferably EINVAL for an
implementation that does not permit malloc(0), ENOMEM if allocation of a
0-size object is permitted but failed, vs. errno left unchanged if it is
even legal to return NULL on success).

It is also odd that whether realloc(x, 0) behaves like malloc(0) or like
free(ptr) depends on whether x was NULL or a valid ptr, where it would
be nicer if there were some consistency.  The fact that realloc(x, 0) is
an obsolescent feature may mean that it's worth having glibc expose a
versioned variant of realloc, one for existing programs compiled against
C11/POSIX 2017 and earlier that keeps current behavior, and one when
compiled against C17 or POSIX Issue 8 and newer that specifically fails
all attempts at realloc(x, 0) with errno set to EINVAL (rather than
treating some like malloc(0) and others like free(x)).

The Austin Group is willing to open another defect to WG14 complaining
that the resolution for defect 400 is inconsistent (in one case it
requires NULL to equate to reporting an error, in another, it still
seems to permit NULL on success, and where the implementation-defined
loophole about whether the pointer is freed or not makes it difficult to
write code that portably handles all permitted C17 behaviors other than
by avoiding the obsolescent feature of passing 0 size to realloc).  But
it would help if we had agreement from glibc developers on whether glibc
has any proposed wording to submit to the C folks, and/or any plans to
use versioned symbols such that glibc realloc(x, 0) with C17 compliance
no longer returns NULL except when also setting errno (either out-right
forbidding realloc(x, 0) with EINVAL, or else treating realloc(x, 0) as
an unconditional synonym for (free(x),malloc(0)) where the only NULL
return is due to ENOMEM), so that it is no longer possible to have to
worry about a NULL pointer meaning success but not having easy
visibility into whether success in turn means that a pointer still needs
freeing.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Attachment: signature.asc
Description: OpenPGP digital signature