[misc intermediate comments removed]
On 11/15/19 3:54 AM, Andrew Luke Nesbit wrote:
In particular I'm trying to figure out a generally applicable way of
taking a
_consistent_ backup of a disk without resorting to single user mode.
I think COW file systems might help in this regard but I don't think
anything like this exists in OpenBSD.
COW in the filesystem, no. However...
a backup is a precautionary copy-before-write.
The only difference is the time granularity.
Consistency? An arbitrary file system snapshot doesn't guarantee that
you won't see -application level- inconsistent data, just that the files
didn't change during backup. Even a COW system that doesn't reveal
a new version of a file until it's been closed won't protect you
from an inconsistent group of files.
What groups of files --must-- be perfectly archived?
If you (a) can pause user activity
(b) can tolerate some inconsistency in captured system log files,
then just run a backup.
Partial DB transactions had better be OK or get a new DBM.
You might have to pause cron(8).
I don't remember any other daemon that would care.
Some archive programs will complain if the size or modification time
of a file changes during copy or if the directory contents change.
Something could be done to automatically go back for those.
Depending very much on your mix of uses, don't even stop anything.
Breaking up the backup into sections - per project, per file system, etc.
can make the pauses less objectionable. It can make recovery easier as well.
Assuming you have control over the system files those only need a couple of
copies when they change, for instance.
Brute force:
ls -Rl /users /proj1 /proj2 > before0
$BACKUP -o /$BACKUPFS/backup-$(date)
ls -Rl /users /proj1 /proj2 > after0
# remove known don't-cares
sed -f ignores before0 > before
sed -f ignores after0 > after
# check to see if any action needed
diff before after > changed
grep -f vitalfiles changed > urgent
cat urgent changed | mail -s "changes during backup $(date)" you
# calculate list of files needing recopy
$SCRIPT changed > newbackup
# copy files missed - should run quickly
$BACKUP -o /$BACKUPFS/bdelta-$(date) -f newbackup
This worked pretty well for me.
The truly paranoid would put a while loop around the diff & recopy...
Binary files can be regenerated if the source *and* environment
are backed up.
Storing the environment is a tricky problem that I haven't found an
entirely satisfactory solution for, yet.
The key is for the project never to use an unqualified program -
always "our current version".
One solution is to copy or link a consistent set of utilities
(compiler, linker, libraries) into the project and always use
those in production. Then a backup will capture everything.
This won't necessarily work if the OS changes its ABI but it
can be pretty effective.
I've been in a project that used this approach and it did work.
Keeping an automatic record of utility and library versions used works as
long as the system itself is backed up well.
The discipline to keep everything tidy, ... well.
Without regard to backups, the precaution to take periodic
snapshots of a project, transplant it into an empty system
and make sure the snapshot actually works
has been erm, revealing.
# mv /usr/bin/cc /usr/bin/saved-cc
# rm /usr/bin/cc
$ make
.....not found <whimper><sob>
Andrew
It can be a pain to design a procedure that fits your needs
and doesn't need a staff of operators (:-(
Good luck!
Geoff Steckel
