Under linux kernel-image-4.19.0-6-amd64 Linux 4.19 for 64-bit PCs (signed) (4.19.67-2 + deb10u1) no problems occur when copying folders with files total volume > 100 Gb.
Problems were found when working under the linux-image kernel-5.3.0-3-amd64 Linux 5.3 for 64-bit PCs (signed) (5.3.15-1) After copying an array of files in 1.4 Tb from one disk to another, by testing copied "*.rar-files" some files are found with partially distorted content. How this is possible is not yet clear. "rar t" on such files gives messages "checksum error". There may not be any such bad files or be from 1 to 6-7 on the entire array of files. And from copy to copy such failures occur in arbitrary files. A total of 4511 rar files of 1.4 Tb were copied. This happens when you copy through: - copy with " Double Commander" - "cp-Rp"../Download" " / media/u1/u-z/U-Z/" - time ionice -c Best-effort-n 7 cp-Rp "../Download" "/media/user/disk2/" In all 3 cases, copying is about 3 or more hours. After that testing the copied files using time ionice -c Best-effort-n 7 rar t-r "*.rar" &> "log.txt" allows you to detect failures that occur when copying rar archives, the message "checksum error" on such files. I. when you test such copied archives, then you find the replacement of entire blocks on the unknown where the dirt has taken, but the size of the files does not change. So at least you can find it. What's going on with files not protected by CRC codes, one can only guess. This shouldn't be happening! I re-copy the failed files manually through "Double Commander" and then test them this archive. And it passes without mistakes. Testing a copied 1.4 Tb archive array takes between 2.5 hours and 3 hours. Then defragment copied through "e4defrag -v 'path to files massive'". Again I check rar-files through time ionice -c Best-effort-n 7 rar t-r "*.rar" &> "log.txt" And then it turns out that again arbitrarily appear archives, which are reported "checksum error". But at the subsequent individual testing of such archives no "checksum error" messages appear. It turned out that after defragment the copied files, you just need to reset the cache: sync; echo 3 > /proc/sys/vm/drop_caches (with administrator rights) And after resetting the cache, start testing the copied archives. Then "checksum error" messages do not appear. When copying individual folders from this array with "*.rar files" there are no such problems. The problem is related to caching information in Debian. The method of "scientific poke" picked up parameters for Seagate ST2000 disks (disks with" Hot-Plug " Barracuda ST2000DMD001, Constellation ES.3 ST2000NM0033 and CS ST2000NC001, not " Hot-Plug» Barracuda ST2000DM008). About the "vm.dirty_background_ratio " look at the installed package "linux-source" in the folder " /usr/src/linux-source-5.3.tar.xz". Open the archive tar-x-f "/usr/src / linux-source-5.3.tar.xz" And look in the folder " linux-source-5.3/Documentation/admin-guide/sysctl/vm.rst". In General, in the folder "linux-source-5.3/Documentation" a lot of very important documentation on configuring Debian. It is necessary to install for Linux kernel-image-5.3.0-3-amd64 Linux 5.3 for 64-bit PCs (signed) (5.3.15-1): In " /etc/sysctl.d/local.conf" vm.dirty_background_ratio = 5 vm.dirty_background_ratio = 10 (default), =9, = 8, =7, and even at =6 when copying through all three copy methods (see above), there are distortion of information when copying from disk to disk. In "Double Commander" are visible temporary slowdown copying and even temporarily repeatedly pausing copying. Using "Double Commander" you can clearly see the change in the speed of copying and the process itself. Interesting, that in the process copying repeatedly were attempts the most Debian to increase the speed of copying, but with the simultaneous further speed drop to 120 Mb/s, although copying started at 170 Mb/s. When "vm.dirty_background_ratio = 5" stopped occurring after copying 1.4 Tb of rar files of the message "checksum error" at "rar t". That is, we can assume that the information is now copied without distortion. On other computers, the acceptable value is " vm.dirty_background_ratio " can be another. When set to " vm.dirty_background_ratio=5 " copying an array of 1.4 Gb files in "Double Commander" takes the same time as in time ionice -c Best-effort-n 7 cp-Rp "../Download" " /media/u1/disk2/" , i.e. about 3 hours and, in the future, when testing the copied information no errors occur.
