[PATCH] io: allow filesystem st_blksize-directed buffer sizes up to 128k instead of 8k

[Pádraig Brady]

On 29/12/2025 02:27, Collin Funk wrote:
> Hi наб,
> 
> наб <nabijaczleweli@nabijaczleweli.xyz> writes:
> 
>> The only difference between old and new being
>>    setvbuf(input, (char *)malloc(128*1024), _IOFBF, 128*1024);
>> with 128k matching zfs's st_blksize:
>>    Benchmark 1: < f  out/cmd/head -n 290955
>>      Time (mean ± σ):     153.4 ms ±   3.9 ms    [User: 78.0 ms, System: 75.3 ms]
>>      Range (min … max):   146.5 ms … 159.2 ms    19 runs
>>
>>    Benchmark 2:  < f ./head.old -n 290955
>>      Time (mean ± σ):     259.7 ms ±   5.7 ms    [User: 86.3 ms, System: 173.3 ms]
>>      Range (min … max):   249.1 ms … 269.3 ms    11 runs
>>
>>    Summary
>>      < f  out/cmd/head -n 290955 ran
>>        1.69 ± 0.06 times faster than  < f ./head.old -n 290955
>> (where the output is 290955 lines, 230427757 bytes and the program
>>   reduces to a getdelim()/fwrite() loop which reduces to a read()/write()
>>   loop).
>>
>> A value around 128k gets us around 60% of the performance back
>> without being so large it should affect the memory pressure.
>>
>> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
>> ---
>>   libio/filedoalloc.c | 11 ++++++++++-
>>   1 file changed, 10 insertions(+), 1 deletion(-)
>>
>> diff --git libio/filedoalloc.c libio/filedoalloc.c
>> index f360d96a9b..3a3d6f52d3 100644
>> --- libio/filedoalloc.c
>> +++ libio/filedoalloc.c
>> @@ -61,6 +61,13 @@
>>   #include <stdlib.h>
>>   #include <unistd.h>
>>   
>> +/* nfs/cifs/cephfs request 512k/1M/4M. That is a /lot/ for an iostream.
>> +   However, many other filesystems request reasonable values,
>> +   and clamping them to BUFSIZ of 8k throws away a lot of performance:
>> +   for a getdelim()/fwrite() loop at zfs's requested st_blksize of 128k,
>> +   over a 220М file, completes in 60% of the time of when the buffer size is 8k. */
>> +#define MAX_FS_BUFSIZ (128 * 1024)
>> +
>>   /* Allocate a file buffer, or switch to unbuffered I/O.  Streams for
>>      TTY devices default to line buffered.  */
>>   int
>> @@ -84,7 +91,9 @@ _IO_file_doallocate (FILE *fp)
>>   	    fp->_flags |= _IO_LINE_BUF;
>>   	}
>>   #if defined _STATBUF_ST_BLKSIZE
>> -      if (st.st_blksize > 0 && st.st_blksize < BUFSIZ)
>> +      if (st.st_blksize > MAX_FS_BUFSIZ)
>> +        st.st_blksize = MAX_FS_BUFSIZ;
>> +      if (st.st_blksize > 0)
>>   	size = st.st_blksize;
>>   #endif
>>       }
> 
> Adding Pádraig and Paul to CC, who certainly know more about reasonable
> buffer sizes than I do.
> 
> However, with some trivial testing I recall doing in Coreutils, large
> buffers don't always have great throughput.
> 
> Here is a trivial example that was run once, so obviously not perfect:
> 
>      $ dd if=/dev/random of=input bs=1G count=10 status=none
>      $ dd iflag=fullblock if=input bs=$(numfmt --from=iec 8k) \
>          status=none | pv -r > /dev/null
>      [ 639MiB/s]
>      $ dd iflag=fullblock if=input bs=$(($(stat -c %o input) * 512)) \
>          status=none | pv -r > /dev/null
>      [ 576MiB/s]
> 
> That test was done on btrfs without any compression, for reference.\

8K does seem like a low clamp, though also glibc would have more
stringent allocation limits than coreutils at least.
Note coreutils defaults to 256KiB buffers, though that is to
minimize system call overhead, and the optimum for a particular
file system may be different. See the notes in:
https://github.com/coreutils/coreutils/blob/master/src/ioblksize.h
cheers,
Padraig