[PATCH] io: allow filesystem st_blksize-directed buffer sizes up to 128k instead of 8k

[наб]

Hi!

On Sun, Dec 28, 2025 at 06:27:15PM -0800, Collin Funk wrote:
> However, with some trivial testing I recall doing in Coreutils, large
> buffers don't always have great throughput.
> 
> Here is a trivial example that was run once, so obviously not perfect:
> 
>     $ dd if=/dev/random of=input bs=1G count=10 status=none
>     $ dd iflag=fullblock if=input bs=$(numfmt --from=iec 8k) \
>         status=none | pv -r > /dev/null
>     [ 639MiB/s]
>     $ dd iflag=fullblock if=input bs=$(($(stat -c %o input) * 512)) \
>         status=none | pv -r > /dev/null
>     [ 576MiB/s]
> 
> That test was done on btrfs without any compression, for reference.

I think the usage pattern of fully-buffered stdio (try to read full block,
process it for a while, read another when you underflow) is a little different
and benefits from bigger blocks a lot more than dd bs=N
which is defined as a straight read(N)/write(N) loop.

Also I think your test is wrong for these two-and-a-half reasons:
1. you're filling an input buffer /512x the size of what you've been recommended/
2. you added iflag=fullblock, which will retry with smaller and smaller
   reads to fill ibs= (which you inherited from bs= in your setup),
   I don't think this is what stdio does?
   I also don't know how it interacts with the btrfs I/O layer
   (does it ever return a short read or do you wait until it fills the
    full buffer, which you wouldn't pay for if you either requested a
    st_blksize-sized buffer or didn't request iflag=fullblock?)
3. you're writing to a default-sized pipe (probably 64k),
   which is serviced with 64k splices by pv
   (dd does see a full-buffer write for me even at 2M,
    but I expect it to be slower than a better reader anyway)

If the conclusion you draw from this is "multi-megabyte buffers are too
big in the general case" then this agrees with my voreutils findings that
summarise to "64k-256k is generally chill, then it starts diminishing".

You didn't say what the st_blksize value is for you
(and git grepping through a kernel checkout didn't get me much more),
but 512*(anything around 4k) is too big (at least 2M)
and I don't think anyone would argue that it isn't.

Best,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/libc-alpha/attachments/20251229/dc37e58b/attachment.sig>