Marvin Poul <pon...@creshal.de> added the comment:
I hope you don't mind me necro posting, but I ran into this issue again and have a small patch to solve it. I attached an MWE that triggers the BlockingIOError reliably on ext4 filesystems in linux 4.12.14 and python 3.8.12. Running under strace -e sendfile gives the following output # manually calling sendfile to check that it works > sendfile(5, 4, [0] => [8388608], 8388608) = 8388608 # sendfile calls originating in shutil.copy > sendfile(5, 4, [0] => [8388608], 8388608) = 8388608 > sendfile(5, 4, [8388608], 8388608) = -1 EAGAIN (Resource temporarily > unavailable) > Shutil Failed! > [Errno 11] Resource temporarily unavailable: > '/cmmc/u/zora/scratch/sendfile_bug/tmpaqx2o4uj' -> > '/cmmc/u/zora/scratch/sendfile_bug/tmpb8rzg8rg' > +++ exited with 0 +++ This shows that the first call to sendfile actually copies the whole file and the EAGAIN is only triggered on the second, unnecessary, call. I have tested with a small C program that it's triggered whenever sendfile's offset + count exceeds the file size of in_fd. This is weird behaviour on the kernels side that seems to have changed in newer kernel versions (issue is not present e.g. on my 5.16.12 laptop). Anyways my patch makes that second call not appear by keeping track of the file size and the bytes written so far. It's against the current python main branch, but if I see correctly this part hasn't changed in years. I have checked the error is not thrown when the patch is applied. (I can only attach one file, so patch is attached in a new one.) ---------- nosy: +pmrv Added file: https://bugs.python.org/file50671/sendfile.py _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue43743> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
