[PATCH] copy_file_range: New function to copy file data

[Florian Weimer]

On 12/18/2017 09:04 PM, Adhemerval Zanella wrote:

> I am seeing a lot of failures from tst-copy_file_range on both x86_64 and i686
> when using the fallback implementation (io/copy_file_range.c) due the fact
> write is returning EFBIG (I attached the test output):

Which file system is this?

> 
> [...]
> openat(AT_FDCWD, "/tmp/tst-copy_file_range-in-mT9I6j", O_RDWR) = 3
> ftruncate(3, 0)                         = 0
> openat(AT_FDCWD, "/tmp/tst-copy_file_range-out-0RbGw2", O_RDWR) = 4
> ftruncate(4, 0)
> [...]
> fcntl(4, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
> read(3, "k2dft\31*b#F=P.AyuQ[\22M\2\37\21f\0243\20\r\177\33Ak"..., 8192) = 8192
> write(4, "k2dft\31*b#F=P.AyuQ[\22M\2\37\21f\0243\20\r\177\33Ak"..., 8192) = -1 EFBIG (File too large)
> lseek(3, -8192, SEEK_CUR)               = 0
> write(1, "tst-copy_file_range.c:285: numer"..., 54tst-copy_file_range.c:285: numeric comparison failure
> [...]

It would be interesting to compare this with the real copy_file_range 
system call.  I don't think it remaps EFBIG, so this might also apply 
there, too.

> I also noted it does not provided a non-LFS version and it a good way forward
> imho, however I think we need to explicit handle the case where a non-LFS
> invocation tries to use copy_file_range in a non-supported way.  For instance
> the snippet:
> 
> [...]
> int fin  = open ("/tmp/file.in",  O_RDWR | O_CREAT | O_TRUNC, 0600);                                                                                                                                                                                                                                                                                          int fout = open ("/tmp/file.out", O_RDWR | O_CREAT | O_TRUNC, 0600);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 char buffer[8192] = { 0xcc };                                                                                                                                                                       const size_t size = 8192;                                                                                                                                                                                                                                                                                                                           pwrite (fin, buffer, size, 0);                                                                                                                                                                                                                                                                                                                                   copy_file_range (fin, 0, fout, &(__off64_t) { INT32_MAX }, size, 0);
> [...]
> 
> Will again return EFBIG.

Hmm.  I think this is just the EFBIG problem.  I'd be more concerned 
about EBADF here.

> We have some options as 1. handle EFBIG
> as an expected retuned error, 2. do not declare copy_file_range for
> !__USE_FILE_OFFSET64, 3. add a dummy implementation for non-LFS
> (which return ENOSYS).

I'd like to reproduce this with the file system you used on a kernel 
with a copy_file_range system call, and see what the system call does there.

>> +  /* Main copying loop.  The buffer size is arbitrary and is a
>> +     trade-off between stack size consumption, cache usage, and
>> +     amortization of system call overhead.  */
>> +  size_t copied = 0;
>> +  char buf[8192];
> 
> Do we you have any numbers with shorter sizes? Maybe

Sorry, could you expand?

Choosing buffer sizes is notoriously difficult, I'm afraid.

Thanks,
Florian