Re: syncing huge files/devices: bmapfs

2023-03-09 Thread Heiko Schlittermann via rsync
Roland via rsync (Sa 11 Feb 2023 20:04:18 CET): > do you have performance comparison vs. plain fail on the same hardware > setup ? I digged a bit more, and the performance issue seems to be due to setup on the receiver's side: btrfs on LVM on dm-crypt on md-raid5 So my bmapfs doesn't solve any p

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Heiko Schlittermann via rsync
Robin Lee Powell via rsync (Di 07 Mär 2023 16:20:39 CET): > First of all, I disagree that rsync handles very large files badly, > but that's not super relevant. :) Consdier --partial for large > files. I'm using --inplace (because the receiving side does snapshots of the underlying file system)

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Heiko Schlittermann via rsync
Hello Kevin, Kevin Korb via rsync (Di 07 Mär 2023 00:01:27 CET): > I am not 100% sure I am interpreting this correctly but I think you are > complaining that the file was being deleted in the first command? If so, > instead of -F try --include='*/' --exclude='*'. Otherwise, maybe you want a > s

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Heiko Schlittermann via rsync
Robin Lee Powell via rsync (Di 07 Mär 2023 07:07:01 CET): > Read the "PER-DIRECTORY RULES AND DELETE" of the man page. (And > don't feel bad, it took me a while to figure it out myself). I did, but left with some uncertainty. "H" hides the files from the transfer? What does it mean? "P" protec

prevent filter excluded files from being deleted

2023-03-06 Thread Heiko Schlittermann via rsync
Hello, given are 2 directories: a ├── a-file └── .rsync-filter b └── a-file I'd like to sync a/ -> b/, but I'd like to *exclude* all files. But I do not want to delete the excluded files. (The real scenario is a way more complex, the above is my reproducer.) and the follow

Re: Can rsync write to a FIFO?

2023-02-11 Thread Heiko Schlittermann via rsync
Chris Green via rsync (Fr 10 Feb 2023 10:31:15 CET): > Can rsync write to a FIFO? Obviously one needs the --inplace to do > this, does one also need --write-devices? I think, there is no point in using rsync, as there is nothing to compare against on the remote side (and I've doubts if rsync wil

syncing huge files/devices: bmapfs

2023-02-09 Thread Heiko Schlittermann via rsync
Hi, recently I had to sync really huge files (VM images), represented either as block devices (on one or both sides) or as regular files. It *seemed* that Rsync didn't work well with block devices (the --copy-devices didn't 'work for me, maybe I'm stupid or it's broken in the Rsync that ships wit

Re: deflate on token returned 0 - rsync error: error in protocol data stream

2021-10-22 Thread Heiko Schlittermann via rsync
Hi, Heiko Schlittermann via rsync (Mi 20 Okt 2021 10:56:51 CEST): > Hello, > > we're using rsync on SLES 12 SP 5 on both sides (for detailed version > info see below) and we're experiencing the following issue on the > sender's side: … > deflate on token retur

deflate on token returned 0 - rsync error: error in protocol data stream

2021-10-20 Thread Heiko Schlittermann via rsync
Hello, we're using rsync on SLES 12 SP 5 on both sides (for detailed version info see below) and we're experiencing the following issue on the sender's side: ``` etc/test/windows2019_x86_64_20210929.gz deflate on token returned 0 (22199 bytes left) rsync error: error in rsync protocol data str

Re: [PATCH] Replace mkproto.pl with mkproto.awk

2019-11-16 Thread Heiko Schlittermann via rsync
Ethan Sommer (So 17 Nov 2019 10:08:16 CST): > > Me? I think, the rsync maintainers, or? > > I just added my 2 cents and would stick with .pl, it has proven to be > > stable :) (Yes, I'm a Perl user.) > Meant to direct that towards the rsync maintainers sorry. Are you > suggesting that something th

Re: [PATCH] Replace mkproto.pl with mkproto.awk

2019-10-31 Thread Heiko Schlittermann via rsync
Ethan Sommer via rsync (Do 31 Okt 2019 17:38:17 CET): > This replaces the build dependency on perl with one on awk which is > already used in the build system and is much more ubiquitous than perl I can't speak for rsync, but nowadays Perl isn't that rare, that a dependeny on it for build purpose

Re: --partial does not "unhide" the files

2018-12-31 Thread Heiko Schlittermann via rsync
Kevin Korb via rsync (Mo 31 Dez 2018 20:50:03 CET): > I can't say I have any idea why rsync would just skip that step and I > can't duplicate it myself. > Your only recourse might be to use --inplace on that system. Hm. receiver.c or rsync.c both contain the debug info string "renaming %s to %s\n

Re: --partial does not "unhide" the files

2018-12-31 Thread Heiko Schlittermann via rsync
Kevin Korb via rsync (So 30 Dez 2018 23:56:44 CET): > I think --partial might be a red herring here. It only applies to what > happens when rsync is aborted in the middle of a file. What happens > without -P? Same happens w/o --partial. I append 2 logs: - a from localhost to remote server, expo

--partial does not "unhide" the files

2018-12-30 Thread Heiko Schlittermann via rsync
Hi, I used --partial to transfer files from my local computer (rsync 3.1.2, Debian) to a remote computer (rsync 3.1.1 WD MyPassport Storage device) The files get transferred, but after successful transfer, the files are not renamed from . to . Where to go next? Here is the verbose output after

Re: Comparison of huge files ineffective?

2018-03-22 Thread Heiko Schlittermann via rsync
Kevin Korb via rsync (Mi 14 Mär 2018 15:25:56 CET): > Do not use --checksum. It has an extremely limited use case. Normally > it is much slower than simply re-copying everything. --checksum means > checksum every file on both ends (even files that only exist on one end) > before doing anything

Re: Comparison of huge files ineffective?

2018-03-14 Thread Heiko Schlittermann via rsync
Kevin Korb via rsync (Mi 14 Mär 2018 14:52:55 CET): > Your observation would be right if you are using --checksum which you > shouldn't be. Otherwise, unless you are using --whole-file rsync will > use its differential algorithm to compare the files. If you are using > --progress you will see it

Comparison of huge files ineffective?

2018-03-14 Thread Heiko Schlittermann via rsync
Hi, how does rsync work if it compares two very huge files on two distinct hosts (rsync uses a networked connection, via SSH)? Some observation seems to indicate, that rsync first reads (and checksums?) the remote (destination) side, then, if finished, it reads (and checksums?) the local (source