--lnk-dest is a nice feature of rsync, however, it would require me to write yet another custom script that so many people already have and reinvent the wheel again. I'm trying to find a ready and time-tested and maintained solution before I go down that path. The rsynch handler in backupninja does pretty much the same thing, but uses cp -l as --link-dest did not exist back when it was written. Nevertheless, I'd prefer block-level deduplication and compression.
Your btrfs setup sounds interesting, Sarunas. Are you running those btrfs volumes on Jessie? Any stability issues? Do you think it could work with just 1GB RAM + 1 GB swap? I was thinking about doing out-of-band deduplication to conserve RAM. Do you use both compression and deduplication (file or block level)? I heard they don't play nicely together. Could you share your mount options? Kind regards, Ondřej Grover On Tue, Oct 20, 2015 at 8:35 PM, Sarunas Burdulis < saru...@math.dartmouth.edu> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 10/20/2015 12:57 PM, Ondřej Grover wrote: > > Hello, > > > > I'm looking for recommendations for backup solutions that don't reinvent > > the wheel and are reliable and used. I want to backup two servers to a > > backup server. The main data content is several hundred GB in many very > > small files. > > > > [...] > > > > I've also looked at the new kids on the block like obnam, attick and > > borgbackup. They look interesting, but I prefer time-tested SW for > backups. > > After realizing that these new backup programs pretty much try to > > replicate features of btrfs or ZFS (incremental snapshots, block-level > > compression and deduplication) I started thinking that I could perhaps > > just send the data to the backup server via rsync and save them to a > > btrfs or ZFS (but the backup server may not have enough RAM for ZFS) and > > create daily snaphosts on the server. If memory will permit (if I > > optimize it), I'd go with ZFS as it should be more reliable. Does > > anybody use such a solution? > > Yes, we do use pretty much the setup you describe, i.e. rsync push to > backup server, btrfs volume with daily snapshot subvolumes on the > server. This particular setup has been in use for more than two years by > now, and is working well with 8-10 servers (totals for all backed-up > filesystems is about 1.2TiB/2.5M files of varying size; transferred > daily delta is of course just a fraction of this). The backup server is > a rather modest self-built Mini-ITX Core i5 with 8GiB RAM and 6TiB > hardware (3ware) RAID-10 storage volume. > > Using btrfs volume in btrfs-raid configuration (raid-1 for data and > metadata) would provide protection against data bit-rot. We use several > such volumes on some of our severs, though not in combination with btrfs > snapshot subvolumes. > > - -- > Sarunas Burdulis > http://math.dartmouth.edu/~sarunas > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQEcBAEBCAAGBQJWJollAAoJEAcPjabqyGA9iD4H/2Hxa4IqYxRRLi55kg8c/tOS > UafxgMMnUB498HHVGMldDZdjqmzBd4djHuVOeJPguu7RaSVKOIBBdpmZdI44LXoj > y3P0m7Ig1cg3kGB7tZTgG6mzsgZQln6F8uW/lNox9EikmWUQ2eQZYIDVbLTEOLXt > pMKVTKQeR1Jdl9HvbIqlPBpniqHn+JvesWUeTDdo47i45jcfRonYfPay0/ua7ql0 > n80ObcOLlYSCw/rKZbkspHz7k9PMEEdeosbkvUkw2ZiGq13E1j6DFwSUTxcrMPhR > Ffd+bL5c0CtKOTVrvtZdahJZ5U/Kl/QG1c78TsJagCN7YTSXRWnDSn6acdDRPmM= > =Edp3 > -----END PGP SIGNATURE----- > >
