This is the mail archive of the
mailing list for the glibc project.
Re: [PATCH] libio: Always use _IO_BUFSIZE for stream buffers [BZ #4099]
- From: Florian Weimer <fweimer at redhat dot com>
- To: Roland McGrath <roland at hack dot frob dot com>
- Cc: GNU C Library <libc-alpha at sourceware dot org>
- Date: Thu, 19 May 2016 16:57:45 +0200
- Subject: Re: [PATCH] libio: Always use _IO_BUFSIZE for stream buffers [BZ #4099]
- Authentication-results: sourceware.org; auth=none
- References: <56E17C8E dot 1070209 at redhat dot com> <20160311215230 dot B5AF32C3C1E at topped-with-meat dot com> <56E69B9D dot 3000808 at redhat dot com> <20160318225258 dot 7D1852C3C60 at topped-with-meat dot com> <56FCF883 dot 4050506 at redhat dot com>
On 03/31/2016 12:14 PM, Florian Weimer wrote:
On 03/18/2016 11:52 PM, Roland McGrath wrote:
Whatever the results, they would not IMHO be relevant here.
POSIX specifies that st_blksize is the "preferred I/O block size for this
object". It's the kernel's responsibility to give userland good advice
through this channel. If there are common buggy kernels that give bad
advice, that is a reason to apply upper and lower limits to the advice from
the kernel. But the expectation should be that the kernel gets fixed to
give good advice, and the optimal thing to do with a good kernel is to
follow its advice.
Since the recommended use of st_blksize in this way is a standard user
feature and not just what stdio's implementation happens to do, there is an
argument to be made that the limiting of the value should be done in the
*stat functions reported st_blksize values rather than in stdio's use of
them. (I'm ambivalent about this point.)
That's a good point. I'll try to get feedback from kernel file system
developers on this matter.
I wasn't able to get any feedback. Based on Rich's point about random
I/O and Roland's earlier suggestion, I'm just capping the reported
buffer size to 8192 in the attached patch.
libio: Limit buffer size to 8192 bytes [BZ #4099]
This avoids overly large buffers with network file systems which report
very large block sizes.
2016-05-19 Florian Weimer <email@example.com>
* libio/filedoalloc.c (_IO_file_doallocate): Limit buffer size to
diff --git a/NEWS b/NEWS
index b3fd3cc..dec9757 100644
@@ -33,6 +33,12 @@ Version 2.24
group: files [SUCCESS=merge] nis
Implemented by Stephen Gallagher (Red Hat).
+* The buffer size for byte-oriented stdio streams is now limited to 8192
+ bytes by default. Previously, on Linux, the default buffer size on most
+ file systems was 4096 bytes (and thus remains unchanged), except on
+ network file systems, where the buffer size was unpredictable and could be
+ as large as several megabytes.
Security related changes:
* An unnecessary stack copy in _nss_dns_getnetbyname_r was removed. It
diff --git a/libio/filedoalloc.c b/libio/filedoalloc.c
index 4f9d738..ded0725 100644
@@ -94,7 +94,7 @@ _IO_file_doallocate (_IO_FILE *fp)
fp->_flags |= _IO_LINE_BUF;
- if (st.st_blksize > 0)
+ if (st.st_blksize > 0 && st.st_blksize < _IO_BUFSIZ)
size = st.st_blksize;