I apologize I had on my list to send you this information since the
Installfest and failed to do so. I'll post it here to share with the
larger audience.
I found the very cheapest storage of backup files you will hopefully
never need turns out to be Google Cloud Platform. Right now I'm only
backing up my Linode server there automatically but I want to get
regularly scheduled backups from other machines too. The cost is like...
20 cents a month for daily backups of most of the important parts of the
system. You can configure the buckets to automatically delete files
over a certain age so you don't even need to script that. HOWEVER, and
this is a little non-intuitive, the cheaper the storage rate you choose,
the longer you must keep the files there, because they want to reduce
transactions. It literally won't let your remove a file from cold
storage that's been there for less than (IIRC) 90 days. Fine by me.
I back up my database and mail directories daily, and do file level
backups of the important directories weekly. In all cases, I tar up the
files, then encrypt them with a password using openssl AES256. Then I
used GCP's gsutil to copy the files into the right bucket. I found
gsutil very easy to use after some initial experimentation. You do need
to do some magic to connect your install to your account so you don't
need to specify credentials with each call but that wasn't too bad.
https://cloud.google.com/storage/docs/gsutil
Being a software engineer, I moved a lot of the functionality into one
script, backuplib.sh, so there's very little duplication between the
scripts, and it all just works once I get it working for one script. I
can post more examples if you want but here are some of the key
functions in that file:
function makeBackupName {
/bin/echo "${BACKUP_DIR}/backup_${1}_${HOSTNAME}_${DATE}.${2}"
}
function makeBucketName {
/bin/echo "gs://dkramer_${HOSTNAME}_backups/"
}
# encryptBackup INPUTFILE OUTPUTFILE
function encryptBackup {
testBackupExists ${1}
/usr/bin/openssl enc -aes-256-cbc -salt -pbkdf2 -in ${1} -out ${2}
-pass env:ENC_PASSWORD
testBackupExists ${2}
/bin/rm ${1}
/bin/chmod 700 ${2}
}
# uploadBackup FILENAME
function uploadBackup {
bucketName=`makeBucketName`
/usr/bin/gsutil cp ${1} "${bucketName}"
echo "Space used:"
gsutil du -sh "${bucketName}"
}
# removeOld DAYS WILDCARD
function removeOld {
/usr/bin/find ${BACKUP_DIR} -name "${2}" -mtime +${1} -delete
}
Given that, the entire script to back up my system files (minus some
other things I do to document the current state of the system):
backup=`makeBackupName system tgz`
backupenc="${backup}.enc"
tar cvzf ${backup} --exclude='var/spool/mail/'
--exclude='var/log/journal' etc home root usr/local var/www var/spool
var/cache var/log
encryptBackup ${backup} ${backupenc}
uploadBackup ${backupenc}
removeOld ${BACKUP_DAYS} 'backup_system_*.tgz.*'
Backing up mail is similar but just under /var/mail, and skipping some
large non-critical files.
Backing up the databases is a little more complicated because my server
is using MySQL for some things and Postgresql in others, and in both
cases I query the list of databases dynamically then generate a dump of
each one, so there's no central list of databases to maintain.
The backups are called by cron of course.
If there's anything you would like me to dig into deeper, let me know.
On 12/14/19 10:17 AM, Jerry Feldman wrote:
I currently have Dropbox professional and Google drive (free). I snapshot
backup my tower to a local hard drive. I'm thinking of backing up the most
recent snapshot to either Dropbox or Google. The advantage of Dropbox is
that it supports Linux so I could either tar or cp or rsync to a folder in
my local Dropbox directory, and it would automatically sync to the Dropbox
cloud. With Google I could use deja dup. I certainly could use aws, but I
already have Dropbox and Google accounts. I'm just looking for opinions on
both.
--
Jerry Feldman <gaf.li...@gmail.com>
Boston Linux and Unix http://www.blu.org
PGP key id: 6F6BB6E7
PGP Key fingerprint: 0EDC 2FF5 53A6 8EED 84D1 3050 5715 B88D 6F6
B B6E7
_______________________________________________
Discuss mailing list
Discuss@lists.blu.org
http://lists.blu.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
Discuss@lists.blu.org
http://lists.blu.org/mailman/listinfo/discuss
