On 03/04/2022 19:37, Frank Steinmetzger wrote:
Am Sun, Apr 03, 2022 at 05:05:07PM +0100 schrieb Wol:
Rsync has a bwlimit argument which helps here. Note that rsync copies
the whole file on what it considers local storage (which can be mounted
network shares) ... this can cause a real slowdown.
It won't help on the initial copy, but look at the - I think it is -
--in-place option.
This one is mostly useful if space on the destination is tight or the data
link (for FS commands) is slooow, because normally rsync creates a new temp
file and moves it into place once the transfer is complete.
And making a temporary file may be exactly what you DON'T want. I make heavy
use of hard links,
… which the -H option is for.
and "make a temp file" absolutely buggers file system integrity ...
Can you elaborate? A temp file is just like any other file. If you do
in-place syncing, then it “buggers up” the original file instead of a temp.
"Make a temp file" breaks hard links. My file system is full of them,
and keeping track of hard links can easily bring a system to its knees
(it brought a previous system of mine to its knees :-)
And my use case with LVM and backups, "make a temp file" does both
absolutely nothing to protect file system integrity
I was not actually talking about file *system* integrity, but that if you
overwrite in-place and the connection is lost during transfer, you are left
with an incomplete file. Hence you just lost data on the destination. The FS
itself is not adversely affected by that, only the file.
If the link is lost (unlikely on my system where both drives are on the
same system) you just start again ... and you haven't actually lost any
data, you've just got a corrupt copy to recover.
and makes every backup waste far more space than is necessary ...
How so? If it’s a COW filesystem, I can understand it (insofar as if you do
in-place, then the existing file is deleted first, which gives the FS more
space for the new copy).
An LVM snapshot *IS* a COW filesystem. And no, --in-place doesn't delete
the original, it just overwrites the bits that have changed.
So my backups, all the identical data is de-duplicated. I have two
separate filesystems I can mount, each of which is a full backup, but
they only take up disk space for "one and a bit".
So --in-place actually has a lot of uses outside your two examples. I have
oodles of space, and both my source and target are on fast sata links in the
same computer, but not using --in-place would be *very* costly for me.
Well I admit I don’t know much outside of my realm of local disks with ext4,
zfs and ssh. While I do run LVM on my laptop, it’s just so that I can have
“partitions” on an otherwise fully encrypted block device. I never used any
of its more fancy features.
Read up on it. Especially if you're running linux, snapshotting root is
a very good idea. I always snapshot root on a friday evening then do my
emerge, and while it's never happened, if it all goes pear-shaped I can
mount my backup and go back to where I was. And if you're doing anything
at all dangerous, snapshotting will protect your "pre" state.
(I only know how to make a snapshot, I've never really investigated
rolling back, but it's nice knowing it's an option :-)
Cheers,
Wol
