posix_fallocate behavior in glibc
Christoph Hellwig
hch@lst.de
Wed Jun 26 06:01:34 GMT 2024
Hi all,
Trond brought the glibc posix_fallocate behavior to my attention.
As a refresher, this is how Open Group defines posix_fallocate:
The posix_fallocate() function shall ensure that any required storage
for regular file data starting at offset and continuing for len bytes
is allocated on the file system storage media. If posix_fallocate()
returns successfully, subsequent writes to the specified file data
shall not fail due to the lack of free space on the file system
storage media.
The glibc implementation in sysdeps/posix/posix_fallocate.c, which is
also by sysdeps/unix/sysv/linux/posix_fallocate.c as a fallback if the
fallocate syscall returns EOPNOTSUPP is implemented by doing single
byte writes at intervals of min(f.f_bsize, 4096).
This assumes the writes to a file guarantee allocating space for future
writes. Such an assumption is false for write out place file systems
which have been around since at least they early 1990s, but are becoming
at lot more common in the last decode. Native Linux examples are
all file systems sitting on zoned devices where this is required
behavior, but also the nilfs2 file system or the LFS mode in f2fs.
On top of that it is fairly common for storage systems exposing
network file system access.
How can we get rid of this glibc fallback that turns the implementations
non-conformant and increases write amplication for no good reason?
More information about the Libc-hacker
mailing list