On 3 Sep 2023 14:20 -0700, from dpchr...@holgerdanske.com (David Christensen): >> 8.07 TiB physically stored on one backup drive holding 174 backups; >> 11.4 TiB total logical (excluding effects of compression) data on the >> source; 7.83 TiB hot current logical data on the source (excluding >> things like ZFS snapshots and compression). > > Without seeing a console session, I am unsure what you mean by "physically > stored", "total logical (excluding effects of compression) data", and "hot > current logical data ... (excluding things like ZFS snapshots and > compression)".
"Physically stored" is how much data, after compression and including file system metadata, is actually written to disk and necessary for all data to be accessible; it's the relevant metric for whether I need to add disk space. "Logical" is the sum of all apparent file sizes as visible to userland utilities e.g. through stat(2). Something like `dd if=/dev/zero of=$(mktemp) bs=1M count=1M` would result in a large logical size but, because of compression, a very small amount of physically stored data. "Hot" is perhaps better referred to as the "current" data set; since snapshots (and earlier backups) can include data which has since been deleted, and is thus no longer current but still exists on disk. > What partitioning scheme, volume manager, file system, compression, etc., do > you use on your backup server? ZFS within LUKS containers. If I recall correctly, the backup pool is set to use zstd compression. > I had thought you were using rsnapsnot/ rsync --link-dest, but you also > mention ZFS snapshots. Please clarify. Mostly ZFS with a rotating snapshot schedule on the source (the root file system is ext4); copied using rsync --link-dest (through rsnapshot) to a ZFS file system which doesn't use snapshots on the backup target. Most of the ZFS file systems are set up to use compression; there are a few where I know _a priori_ that the data is in effect completely incompressible so there's no point in using CPU to even try to compress that data, so those have compression turned off. (In ZFS, creating a file system is barely any more involved than creating a directory, and all file systems come out of the same "pool" which is a collection of >=1 storage devices set up with some particular method of redundancy, possibly none. In more traditional *nix parlace, a *nix file system is conceptually closer to a ZFS pool.) Hopefully this is more clear. -- Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?”
