This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH] copy_file_range: New function to copy file data
- From: Florian Weimer <fweimer at redhat dot com>
- To: Adhemerval Zanella <adhemerval dot zanella at linaro dot org>, libc-alpha at sourceware dot org
- Date: Mon, 18 Dec 2017 21:19:33 +0100
- Subject: Re: [PATCH] copy_file_range: New function to copy file data
- Authentication-results: sourceware.org; auth=none
- References: <20171117145604.655D24469B999@oldenburg.str.redhat.com> <16de12bf-eab1-b69c-cd58-a6dd24ad409f@redhat.com> <eeee1a5f-63c5-4379-1fd4-7f0319d28cd4@linaro.org>
On 12/18/2017 09:04 PM, Adhemerval Zanella wrote:
I am seeing a lot of failures from tst-copy_file_range on both x86_64 and i686
when using the fallback implementation (io/copy_file_range.c) due the fact
write is returning EFBIG (I attached the test output):
Which file system is this?
[...]
openat(AT_FDCWD, "/tmp/tst-copy_file_range-in-mT9I6j", O_RDWR) = 3
ftruncate(3, 0) = 0
openat(AT_FDCWD, "/tmp/tst-copy_file_range-out-0RbGw2", O_RDWR) = 4
ftruncate(4, 0)
[...]
fcntl(4, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
read(3, "k2dft\31*b#F=P.AyuQ[\22M\2\37\21f\0243\20\r\177\33Ak"..., 8192) = 8192
write(4, "k2dft\31*b#F=P.AyuQ[\22M\2\37\21f\0243\20\r\177\33Ak"..., 8192) = -1 EFBIG (File too large)
lseek(3, -8192, SEEK_CUR) = 0
write(1, "tst-copy_file_range.c:285: numer"..., 54tst-copy_file_range.c:285: numeric comparison failure
[...]
It would be interesting to compare this with the real copy_file_range
system call. I don't think it remaps EFBIG, so this might also apply
there, too.
I also noted it does not provided a non-LFS version and it a good way forward
imho, however I think we need to explicit handle the case where a non-LFS
invocation tries to use copy_file_range in a non-supported way. For instance
the snippet:
[...]
int fin = open ("/tmp/file.in", O_RDWR | O_CREAT | O_TRUNC, 0600); int fout = open ("/tmp/file.out", O_RDWR | O_CREAT | O_TRUNC, 0600); char buffer[8192] = { 0xcc }; const size_t size = 8192; pwrite (fin, buffer, size, 0); copy_file_range (fin, 0, fout, &(__off64_t) { INT32_MAX }, size, 0);
[...]
Will again return EFBIG.
Hmm. I think this is just the EFBIG problem. I'd be more concerned
about EBADF here.
We have some options as 1. handle EFBIG
as an expected retuned error, 2. do not declare copy_file_range for
!__USE_FILE_OFFSET64, 3. add a dummy implementation for non-LFS
(which return ENOSYS).
I'd like to reproduce this with the file system you used on a kernel
with a copy_file_range system call, and see what the system call does there.
+ /* Main copying loop. The buffer size is arbitrary and is a
+ trade-off between stack size consumption, cache usage, and
+ amortization of system call overhead. */
+ size_t copied = 0;
+ char buf[8192];
Do we you have any numbers with shorter sizes? Maybe
Sorry, could you expand?
Choosing buffer sizes is notoriously difficult, I'm afraid.
Thanks,
Florian