On 03/02/2024 16:02, Grant Edwards wrote:
rsnapshot is an application that uses rsync to do
hourly/daily/weekly/monthly (user-configurable) backups of selected
directory trees. It's done using rsync to create snapshots. They are
in-effect "incremental" backups, because the snapshots themselves are
effectively "copy-on-write" via clever use of hard-links by rsync. A
year's worth of backups for me is 7 daily + 4 weekly + 12 monthly
snapshots for a total of 23 snapshots. If nothing has changed during
the year, those 23 snapshots take up the same amount of space as a
single snapshot.
So as I understand it, it looks like you first do a "cp with hardlinks"
creating a complete new directory structure, but all the files are
hardlinks so you're not using THAT MUCH space for your new image?
Then rsync copies and replaces any files that have been modified?
So each snapshot is using the space required by the directory structure,
plus the space required by any changed files.
My understanding of ZFS is that it has built-in snapshot functionality
that provides something similar to what is done by rsync by its use of
hard-links. In my current setup, there's an application called
rsnapshot that manages/controls the snapshots by invoking rsync in
various ways. My question was about the existence of a similar
application that can be used with ZFS's built-in snapshot support to
provide a similar backup scheme.
ZFS is a "copy on write" filesystem, I believe. So any changed blocks
are rewritten to new blocks, the live snapshot is updated, and if the
original block is required by older snapshots it is retained, else it is
freed. And then you can push a snapshot to another disk. I think ZFS
also has "dedupe on write", so if you're regularly pushing snapshots
you're not wasting space with duplicate data.
And that is why I like "ext over lvm copying with rsync" as my strategy
(not that I actually do it). You have lvm on your backup disk. When you
do a backup you do "rsync with overwrite in place", which means rsync
only writes blocks which have changed. You then take an lvm snapshot
which uses almost no space whatsoever.
So to compare "lvm plus overwrite in place" to "rsnapshot", my strategy
uses the space for an lvm header and a copy of all blocks that have changed.
Your strategy takes a copy of the entire directory structure, plus a
complete copy of every file that has changed. That's a LOT more.
If my hard disk changes by lets say 0.1% a day, and I take daily
snapshots, that's three years before I need to start deleting backups
assuming I'm actually using half my disk (and with terabyte disks, both
the amount of change, and the amount of disk used, is likely to be a lot
less than those figures).
Cheers,
Wol
