Bug 13151 - fmemopen streams fail to read eof
Summary: fmemopen streams fail to read eof
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: stdio (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-09-05 02:24 UTC by Rich Felker
Modified: 2015-07-08 15:16 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rich Felker 2011-09-05 02:24:22 UTC
Streams obtained by fmemopen fail to yield EOF when attempting to read past the "current size" of the buffer. Minimal failure case:

FILE *f = fmemopen((char[10]){"hello"}, 10, "a+");
assert(getc(f)==EOF);

(The "current size" established by opening in append mode is the offset of the null byte, 5.)

Reference:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/fmemopen.html

I understand that the GNU fmemopen predated the standardized function, and I'm not sure what politics were involved in writing the standard such that the existing GNU version did not conform. Nonetheless, this should be fixed. If you want to keep the original GNU behavior, there should be a separate __posix_fmemopen that gets used when _POSIX_C_SOURCE >= 200809L.
Comment 1 Michael Kerrisk 2012-04-27 20:15:20 UTC
I think your example isn't quite right, but there is a real problem here.

See the fopen(3) man page:

       a+     Open for reading and appending (writing at end  of
              file).   The file is created if it does not exist.
              The initial file position for reading  is  at  the
              beginning  of  the  file,  but  output  is  always
              appended to the end of the file.

So, "a+" should set the file pointer to byte 0.

It looks like the problem is that the file position is not correctly reset to 0 when mode is "a+". Without a command-line argument, the program below does an fmemopen(), and then tries reading characters, and we see the problem:

$ ./a.out
Initial ftell(): -1
Segmentation fault (core dumped)

With a command-line argument, the program does an initial fseek() to explicitly place the file pointer at byte 0, and then all is well:

./a.out x
Initial ftell(): -1
Resetting file position
New ftell(): 0
 0: h 104; feof()=0; ftell() = 1
 1: e 101; feof()=0; ftell() = 2
 2: l 108; feof()=0; ftell() = 3
 3: l 108; feof()=0; ftell() = 4
 4: o 111; feof()=0; ftell() = 5
 5: � -1; feof()=1; ftell() = 5

So, it seems that the fix would be that when fmemopen() mode is "a+", the glibc implementation should set the file position to 0.

=========
#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#define errExit(msg) 	do { perror(msg); exit(EXIT_FAILURE); \
			} while (0)

int
main(int argc, char **argv)
{
    FILE *fp = fmemopen((char[20]){"hello, world"}, 5, "a+");
    char c;
    int j;

    printf("Initial ftell(): %ld\n", ftell(fp));

    if (argc > 1) {
	printf("Resetting file position\n");
        fseek(fp, 0, SEEK_SET);
        printf("New ftell(): %ld\n", ftell(fp));
    }

    for (j = 0; j < 20 && !feof(fp); j++) {
        c = fgetc(fp);
	printf("%2d: %c %d; feof()=%d; ftell() = %ld\n",
		j, c, c, feof(fp), ftell(fp));
    }
}
Comment 2 Michael Kerrisk 2012-04-27 20:31:07 UTC
Rich,

I spoke a little soon. I see that that the fmemopen() POSIX spec has a definition of "a+" that is more consistent with your explanation. Anyway, we both agree there's a problem here ;-).
Comment 3 Michael Kerrisk 2012-04-28 02:20:55 UTC
I tested a little more. See the code below. 

The problem seems to occur only when the specified size of the buffer does not include a null byte. If the specified size does include a null byte, then the file position is correctly set to the first null byte. The standard does clearly cover both cases:

[[
The stream maintains a current position in the buffer. This position is initially set to either the beginning of the buffer (for r and w modes) or to the first null byte in the buffer (for a modes). If no null byte is found in append mode, the initial position is set to one byte after the end of the
buffer.
]]

Thus, glibc's fmemopen() should be setting the file position to the byte after the end of the supplied string. Instead, it sets the file position to -1.

Using the test program below, here's what happens when the specified size includes the null byte:

$ ./a.out 15
Initial ftell(): 12

And here's what happens if the specified size doesn't include the null byte:

$ ./a.out  10
Initial ftell(): -1



/* fmemopen_a+_file_pos_bug.c
*/
#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>

#define errExit(msg) 	do { perror(msg); exit(EXIT_FAILURE); \
			} while (0)

#define BSIZE 20
int
main(int argc, char **argv)
{
    char buf[BSIZE] = {"hello, world"};
    FILE *fp;

    if (argc < 2) {
	fprintf(stderr, "Usage: %s <nchars> [fseek-pos]\n", argv[0]);
	exit(EXIT_FAILURE);
    }

    fp = fmemopen(buf, atoi(argv[1]), "a+");
    if (fp == NULL)
	errExit("fmemopen");

    printf("Initial ftell(): %ld\n", ftell(fp));

    if (argc > 2) {
	printf("Resetting file position\n");
        fseek(fp, atoi(argv[2]), SEEK_SET);
        printf("New ftell(): %ld\n", ftell(fp));
    }
}
Comment 4 Michael Kerrisk 2012-04-28 03:27:37 UTC
I've added the following text to the man page under BUGS

[[
If the
.I size
argument given to
.BR fmemopen ()
does not cover a null byte in
.IR buf
then, according to POSIX.1-2008,
the file position should be set to the next byte after the end of the buffer.
However, in this case, the glibc
.\" FIXME http://sourceware.org/bugzilla/show_bug.cgi?id=13151
.BR fmemopen ()
sets the file position to \-1.
]]
Comment 5 Michael Kerrisk 2012-04-28 04:26:34 UTC
Twaeked that man page text. Now it is:

If the
.I mode
argument to
.BR fmemopen ()
specifies append ("a" or "a+"), and the
.I size
argument does not cover a null byte in
.IR buf
then, according to POSIX.1-2008,
the initial file position should be set to
the next byte after the end of the buffer.
However, in this case the glibc
.\" FIXME http://sourceware.org/bugzilla/show_bug.cgi?id=13151
.BR fmemopen ()
sets the file position to \-1.
Comment 6 OndrejBilka 2013-05-20 15:24:11 UTC
Michael,
I am refactoring fmemopen. It should handle a+ mode like fopen does. See
http://sourceware.org/ml/libc-alpha/2013-05/msg00729.html

If I read correctly should be also resolved by this patch.
Comment 7 Jackie Rosen 2014-02-16 18:29:02 UTC Comment hidden (spam)
Comment 8 Adhemerval Zanella 2015-07-08 15:16:18 UTC
This is fixed by fdb7d390dd0d96e4a8239c46f3aa64598b90842b.  Closing bug.