Your observation would be right if you are using --checksum which you shouldn't be. Otherwise, unless you are using --whole-file rsync will use its differential algorithm to compare the files. If you are using --progress you will see it step through the file at a faster speed than the network is capable of. If there is no difference then rsync will go very fast to 100% then do any metadata changes. If it hits a difference and you aren't using --inplace then rsync will copy the correct beginning of the file to a new temp file then begin transferring the differences. On huge files this can cause a significant pause while it does that local copy to a temp file.
On 03/14/2018 09:25 AM, Heiko Schlittermann via rsync wrote:
> Hi,
>
>
> how does rsync work if it compares two very huge files on two distinct
> hosts (rsync uses a networked connection, via SSH)?
>
> Some observation seems to indicate, that rsync first reads (and
> checksums?) the remote (destination) side, then, if finished, it reads
> (and checksums?) the local (source) side and then starts transferring
> the delta if done with the comparison of these two files on both sides.
>
> The major problem (with respect to runtime) seems to be the missing
> parellism in read (and checksumming) the source and the destination.
>
> 1) Is this observation right?
> 1a) Does it depend on the rsync version?
>
> 2) Is there a way to get read (and checksumming) of both sides
> in parallel?
>
>
> Best regards from Dresden/Germany
> Viele Grüße aus Dresden
> Heiko Schlittermann
>
>
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. ke...@futurequest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
signature.asc
Description: OpenPGP digital signature
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
