Bug 11216

Summary: fmemopen returns NULL when len=0
Product: glibc Reporter: Alex Shinn <alexshinn>
Component: stdioAssignee: Ulrich Drepper <drepper.fsp>
Status: RESOLVED FIXED    
Severity: normal CC: adhemerval.zanella, bugdal, eblake, glibc-bugs, lynneandallan, msebor, mtk.manpages
Priority: P2 Flags: fweimer: security-
Version: 2.10   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:

Description Alex Shinn 2010-01-24 11:35:52 UTC
The following program prints "(nil)" demonstrating the problem:

  int main (int argc, char **argv) {
    char buf[] = "";
    FILE *in = fmemopen(buf, 0, "r");
    fprintf(stderr, "%p\n", in);
  }

As a general-purpose utility for converting arbitrary strings
to streams it makes no more sense to forbid empty strings
(which would simply return EOF immediately) than it does to
forbid empty files or /dev/null.

[However, since it doesn't make sense to have an output stream
with an empty buffer, it seems reasonable to continue to return
NULL for the "w"rite case.]

In the meantime, as a workaround in my own applications I will
wrap fmemopen to open /dev/null in this case.
Comment 1 Dmitry V. Levin 2010-01-24 21:49:55 UTC
(In reply to comment #0)
> [However, since it doesn't make sense to have an output stream
> with an empty buffer, it seems reasonable to continue to return
> NULL for the "w"rite case.]

It might have a sense, too: every write would fail with ENOSPC, like every write
to /dev/full does.
Comment 2 Ulrich Drepper 2010-02-04 12:05:10 UTC
You want an existing interface to be changed just to fit what you think is best?
 Regardless of the impact on existing applications?

There won't be any change.  The implementation is as it is.  Perhaps not optimal
but nobody can prove there are no applications which depend on the existing
semantic.
Comment 3 Rich Felker 2011-09-05 02:06:05 UTC
Nowadays fmemopen is specified by POSIX 2008, and POSIX 2008 has no language that it shall or may fail if the size is 0. I think this bug should be reconsidered.
Comment 4 Michael Kerrisk 2012-04-28 02:29:48 UTC
Adding Eric Blake to the CC, since he may be interested in this bug from a POSIX perspective.
Comment 5 Michael Kerrisk 2012-04-28 03:21:41 UTC
I've added the following text to the fmemopen(3) man page:

[[
If
.I size
is specified as zero,
.BR fmemopen ()
fails with the error
.BR EINVAL .
.\" FIXME http://sourceware.org/bugzilla/show_bug.cgi?id=11216
It would be more consistent if this case successfully created
a stream that then returned end of file on the first attempt at reading.
]]
Comment 6 paxdiablo 2014-01-30 03:38:41 UTC
I'm not entirely sure why is this being considered a bug.

The 2013 publication of POSIX.1 at http://pubs.opengroup.org/onlinepubs/9699919799/functions/fmemopen.html specifically states (in the return value section):

===
The fmemopen() function shall fail if:
[EINVAL]
    The size argument specifies a buffer size of zero.
===

Hence returning NULL seems to be the RIGHT thing to do.

People may well believe it's more consistent to allow it and fail on first read, but that's not what POSIX says to do. "Shall" in standards context means "must", there's no room for movement there unless you want to claim non-compliance with said standard :-)
Comment 7 Eric Blake 2014-01-30 04:04:22 UTC
(In reply to paxdiablo from comment #6)
> I'm not entirely sure why is this being considered a bug.

POSIX defined fmemopen using glibc as its reference implementation - we could just as easily argue that glibc is correct and POSIX is in error for requiring failure on len=0.

> People may well believe it's more consistent to allow it and fail on first
> read, but that's not what POSIX says to do. "Shall" in standards context
> means "must", there's no room for movement there unless you want to claim
> non-compliance with said standard :-)

Or claim that said standard was incorrect in what it requires.  I've gone ahead and opened http://austingroupbugs.net/view.php?id=818 - let's wait until that has an answer from the POSIX folks before deciding what to do here.
Comment 8 Eric Blake 2014-01-30 04:09:58 UTC
While we're at it, the POSIX folks have a question for glibc:
http://austingroupbugs.net/view.php?id=657
http://sourceware.org/bugzilla/show_bug.cgi?id=15298

The glibc behavior is not very well documented nor very intuitive, and POSIX is stuck trying to determine if glibc and/or the standard needs updating.  How much room do we have for improving things?
Comment 9 paxdiablo 2014-01-30 04:23:36 UTC
> POSIX defined fmemopen using glibc as its reference implementation - we could just as easily argue that glibc is correct and POSIX is in error for requiring failure on len=0.

Except for the near-fatal flaw in your contention, that _POSIX_ is the standard, not glibc :-)

> Or claim that said standard was incorrect in what it requires.  I've gone ahead
and opened http://austingroupbugs.net/view.php?id=818 - let's wait until that
has an answer from the POSIX folks before deciding what to do here.

No probs. My view is that it's set in stone now until they change it in a future edition. There's no real getting around the "shall" language in the _very_ specific text of the standard dealing with the return code, that's not something that slips in inadvertently, it was a very conscious decision.

So, as long as we're clear this isn't (pending POSIX resolution of the issue you raised) currently a bug in glibc itself (and someone's not going to go off half-cocked and "fix" it), it's probably best to leave it open.
Comment 10 Eric Blake 2014-03-13 16:30:32 UTC
Hooray! POSIX relaxed their spec today.  Unless objections are raised before the 30-day review completes, the pre-POSIX behavior of treating a 0 length buffer as a valid empty file similar to /dev/null is now permitted for Issue 7, and future directions says it will probably even be required in Issue 8.
Comment 11 Eric Blake 2014-03-13 16:37:52 UTC
(In reply to Eric Blake from comment #10)
> Hooray! POSIX relaxed their spec today.

http://austingroupbugs.net/view.php?id=818#c2184
"
Delete line 29196.

Add a new paragraph after 29200:

    [EINVAL] The size argument specifies a buffer size of zero and the implementation does not support this.


Add to Future Directions after line 29235

A future revision of this standard may require support of zero length buffer streams explicitly.
"

Basically, the shall fail was relaxed to a mail fail, with the intent of removing the failure altogether and instead requiring successful use of a 0-length memstream, back to the original glibc behavior.
Comment 12 Adhemerval Zanella 2015-07-08 15:13:44 UTC
With this latest clarification and recent new fmemopen implementation (fdb7d390dd0d96e4a8239c46f3aa64598b90842b) I am closing this bug.

If and when the spec changes to 'shall not fail' we can reopen or create another tracking BZ.
Comment 13 Andreas Schwab 2015-08-03 14:10:54 UTC
*** Bug 18756 has been marked as a duplicate of this bug. ***