Hello all, I'm in need of some help in trying to understand rsync's behavior while doing --checksum. I'm rsyncing (dry run) locally from two different folders lots of big files (mainly for consistency checks). Without --checksum, the operation takes around 1 minute.
Running -vvv I see basically this: 1. rsync [sender] traverses a lot of files *but not all* and outputs make_file(... This takes a long time 2. then 40-50 mins later.. a new process seems to be awoken 3. rsync [generator] then does make_file(... for the first file which were traversed during step 1 4. rsync does recv_generator, send_files and itemizes (output) for the first file 5. some sequences of recv_file_list done.. 6. final recv_files for the first file Steps 3-6 are done within 1-2 seconds. Example: 2020/01/03 10:26:53 [31072] [sender] make_file(file.big,*,2) 2020/01/03 11:13:12 [31072] [generator] make_file(file.big,*,2) 2020/01/03 11:13:12 [31072] recv_generator(file.big,64) 2020/01/03 11:13:12 [31072] send_files(64, /folder/file.big) 2020/01/03 11:13:12 [31072] .f file.big 2020/01/03 11:13:13 [31072] recv_files(file.big) What is going on exactly? When is the checksum computed at the source and when at the target? Why the 40-50 mins between steps 1 and 2? are there some kind of batching being done? I've read from the doc the following paragraphs: "The sending side generates its checksums while it is doing the file-system scan that builds the list of the available files. The receiver generates its checksums when it is scanning for changed files, and will checksum any file that has the same size as the corresponding sender's file: files with either a changed size or a changed checksum are selected for transfer" ..but I don't quite put all the pieces together. rsync (3.0.9) is ran with flags: -ahv -W --timeout=14400 -i --delete --chmod=+r -n --checksum -vv --backup ..and some --exclude Thanks a lot! -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
