I could try to request a RAM upgrade (not super easy in uni environment, but doable), but it goes against my understanding of what a storage server is. In my book, a storage server needs fast IO capabilities (storage drives and NIC) and possibly CPU for compression. Deduplication is the only reason I could think of higher RAM requirements on a storage server.
Sarunas, did you have good experiences with actually using the backups for restoring? On Wed, Oct 21, 2015 at 4:28 PM, Sarunas Burdulis < saru...@math.dartmouth.edu> wrote: > On 10/21/2015 05:32 AM, Ondřej Grover wrote: > > [...] > > Your btrfs setup sounds interesting, Sarunas. Are you running those > > btrfs volumes on Jessie? Any stability issues? Do you think it could > > work with just 1GB RAM + 1 GB swap? I was thinking about doing > > out-of-band deduplication to conserve RAM. Do you use both compression > > and deduplication (file or block level)? I heard they don't play nicely > > together. Could you share your mount options? > > Our backup server is actually running Linux 3.13.0-66-generic, Btrfs > v3.12 — Ubuntu 14.04.3 LTS, which is behind Jessie in software versions. > It has been very stable — for the past too years it's running as a > headless box in an off-site location. > > We don't use compression or deduplication. In-band dedup is not in btrfs > 3.*, and I haven't tried any out-of-band tools. Nothing particular in > Btrfs mount, the only non-default options in 'noatime': > > LABEL=BACKUPS /backups btrfs noatime 0 0 > > 1GB RAM sounds low by today's habits. That doesn't mean it won't work, > but just begs for RAM upgrade. > > Sarunas > > > On Tue, Oct 20, 2015 at 8:35 PM, Sarunas Burdulis > > <saru...@math.dartmouth.edu <mailto:saru...@math.dartmouth.edu>> wrote: > > > > On 10/20/2015 12:57 PM, Ondřej Grover wrote: > >> Hello, > > > >> I'm looking for recommendations for backup solutions that don't reinvent > >> the wheel and are reliable and used. I want to backup two servers to a > >> backup server. The main data content is several hundred GB in many very > >> small files. > > > >> [...] > > > >> I've also looked at the new kids on the block like obnam, attick and > >> borgbackup. They look interesting, but I prefer time-tested SW for > backups. > >> After realizing that these new backup programs pretty much try to > >> replicate features of btrfs or ZFS (incremental snapshots, block-level > >> compression and deduplication) I started thinking that I could perhaps > >> just send the data to the backup server via rsync and save them to a > >> btrfs or ZFS (but the backup server may not have enough RAM for ZFS) and > >> create daily snaphosts on the server. If memory will permit (if I > >> optimize it), I'd go with ZFS as it should be more reliable. Does > >> anybody use such a solution? > > > > Yes, we do use pretty much the setup you describe, i.e. rsync push to > > backup server, btrfs volume with daily snapshot subvolumes on the > > server. This particular setup has been in use for more than two years by > > now, and is working well with 8-10 servers (totals for all backed-up > > filesystems is about 1.2TiB/2.5M files of varying size; transferred > > daily delta is of course just a fraction of this). The backup server is > > a rather modest self-built Mini-ITX Core i5 with 8GiB RAM and 6TiB > > hardware (3ware) RAID-10 storage volume. > > > > Using btrfs volume in btrfs-raid configuration (raid-1 for data and > > metadata) would provide protection against data bit-rot. We use several > > such volumes on some of our severs, though not in combination with btrfs > > snapshot subvolumes. > > > > -- > > Sarunas Burdulis > > http://math.dartmouth.edu/~sarunas > >
