[PATCH] io: allow filesystem st_blksize-directed buffer sizes up to 128k instead of 8k

наб nabijaczleweli@nabijaczleweli.xyz
Sun Dec 28 20:23:38 GMT 2025


The only difference between old and new being
  setvbuf(input, (char *)malloc(128*1024), _IOFBF, 128*1024);
with 128k matching zfs's st_blksize:
  Benchmark 1: < f  out/cmd/head -n 290955
    Time (mean ± σ):     153.4 ms ±   3.9 ms    [User: 78.0 ms, System: 75.3 ms]
    Range (min … max):   146.5 ms … 159.2 ms    19 runs

  Benchmark 2:  < f ./head.old -n 290955
    Time (mean ± σ):     259.7 ms ±   5.7 ms    [User: 86.3 ms, System: 173.3 ms]
    Range (min … max):   249.1 ms … 269.3 ms    11 runs

  Summary
    < f  out/cmd/head -n 290955 ran
      1.69 ± 0.06 times faster than  < f ./head.old -n 290955
(where the output is 290955 lines, 230427757 bytes and the program
 reduces to a getdelim()/fwrite() loop which reduces to a read()/write()
 loop).

A value around 128k gets us around 60% of the performance back
without being so large it should affect the memory pressure.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
---
 libio/filedoalloc.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git libio/filedoalloc.c libio/filedoalloc.c
index f360d96a9b..3a3d6f52d3 100644
--- libio/filedoalloc.c
+++ libio/filedoalloc.c
@@ -61,6 +61,13 @@
 #include <stdlib.h>
 #include <unistd.h>
 
+/* nfs/cifs/cephfs request 512k/1M/4M. That is a /lot/ for an iostream.
+   However, many other filesystems request reasonable values,
+   and clamping them to BUFSIZ of 8k throws away a lot of performance:
+   for a getdelim()/fwrite() loop at zfs's requested st_blksize of 128k,
+   over a 220М file, completes in 60% of the time of when the buffer size is 8k. */
+#define MAX_FS_BUFSIZ (128 * 1024)
+
 /* Allocate a file buffer, or switch to unbuffered I/O.  Streams for
    TTY devices default to line buffered.  */
 int
@@ -84,7 +91,9 @@ _IO_file_doallocate (FILE *fp)
 	    fp->_flags |= _IO_LINE_BUF;
 	}
 #if defined _STATBUF_ST_BLKSIZE
-      if (st.st_blksize > 0 && st.st_blksize < BUFSIZ)
+      if (st.st_blksize > MAX_FS_BUFSIZ)
+        st.st_blksize = MAX_FS_BUFSIZ;
+      if (st.st_blksize > 0)
 	size = st.st_blksize;
 #endif
     }
-- 
2.39.5
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20251228/f3399f60/attachment.sig>


More information about the Libc-alpha mailing list