Re: Backup scripts - recycling old backup directories (Kevin Korb)

2014-09-16 Thread Perry Hutchison
Robert Bell wrote: > > 2. Metadata history. If there is an existing file in the target dir > > that differs only by metadata (permissions, ownership, timestamp) then > > rsync will simply change that metadata. That change affects all > > instances of that file. Of course this is better for sto

Re: Backup scripts - recycling old backup directories (Kevin Korb)

2014-09-15 Thread Paul Slootman
On Mon 15 Sep 2014, Kevin Korb wrote: > > I would never operate in a manner that only has 5-6 days of old > backups. The backups that I am deleting are more than a year old. I keep the Sunday backups for a month, the 1st of the month backups for a year. The other daily backups are expired after

Re: Backup scripts - recycling old backup directories (Kevin Korb)

2014-09-15 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I would never operate in a manner that only has 5-6 days of old backups. The backups that I am deleting are more than a year old. On 09/15/2014 02:24 AM, Robert Bell wrote: > Kevin, > > Thanks for the reply and interest in this topic. > > Comments

Re: Backup scripts - recycling old backup directories (Kevin Korb)

2014-09-14 Thread Robert Bell
Kevin, Thanks for the reply and interest in this topic. Comments below. Regards Rob. I did consider that but rejected it for 2 reasons... 1. Backup run time. We have a 4 hour window to run backups at night. Using recycled directories significantly extended the backup run time. The deleti

Re: Backup scripts - recycling old backup directories

2014-09-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I did consider that but rejected it for 2 reasons... 1. Backup run time. We have a 4 hour window to run backups at night. Using recycled directories significantly extended the backup run time. The deletion time is eliminated but frankly, we have th

Re: Backup scripts - recycling old backup directories

2014-09-12 Thread Brian Cuttler
I'm using rsync to backup file systems from one machine to another (not sure why we aren't using amanda like we are everywhere else). The combination of rsync updating blocks, rather than replacing files or replicating file systems is a huge savings for us. As daily zfs snapshots (saved for a day

Backup scripts - recycling old backup directories

2014-09-11 Thread Robert Bell
Folks, Kevin Korb wrote: Have you considered more advanced methods such as subvolume snapshots provided by ZFS and BTRFS? At work we were forced to abandon rsync - --link-dest because of the amount of time it takes to delete old backups when the data is primarily many millions of small files (s

Re: Backup scripts

2014-09-09 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Have you considered more advanced methods such as subvolume snapshots provided by ZFS and BTRFS? At work we were forced to abandon rsync - --link-dest because of the amount of time it takes to delete old backups when the data is primarily many million

Re: Backup scripts

2014-09-09 Thread Henri Shustak
Hello, The --link-dest approach is the way LBackup works and this is also what allows the lcd command (part of LBackup) to move you back and forth in time while you remain in the same directory relative directory. This makes finding a particular data quite easy during a partial restore. In par

Re: Backup scripts

2014-09-04 Thread Henri Shustak
Hello, You may want to check out the LBackup source code as a possible starting point if you are looking to create your own customised backup script : http://www.lbackup.org/source Disclaimer : I am involved with the development of LBackup. -

Re: Backup scripts

2014-09-04 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 That is how to do an incremental backup. But rsync can do a snapshot with --link-dest instead of --backup. The big difference is that with - --link-dest each backup will contain the entire tree not just the different files. Therefore if you need to

Re: Backup scripts

2014-09-04 Thread Greg Deback (rsync)
Hi, I think the words used are not correct, that's why you (and Kevin) are a bit confused. You are in fact asking for "incremental" backup, not a "snapshot" : http://en.wikipedia.org/wiki/Snapshot_(computer_storage). In this case, you can use rsync with a few flags to delete (or move in another di

Re: Backup scripts

2014-09-04 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Just to show you what rsync snapshots should look like here is a really quick and dirty script that will make a complete backup then a snapshot using most of the information from your scripts: http://dpaste.com/27V2WAF Note that it just makes 2 backup

Re: Backup scripts

2014-09-04 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, you have different scripts now at least. But I still don't see any snapshotting. The only difference in these is that the "full" one rsyncs to a date+time stamped directory while the "snapshot" one rsyncs to a directory named snapshot. As far a

Re: Backup scripts

2014-09-04 Thread Chris
On Thu, 2014-09-04 at 17:00 -0400, Kevin Korb wrote: > Do you actually have any snapshots currently? From the scripts you > posted it seems to just be rsyncing to the same dir every run and only > claiming to be making snapshots. If you do have snapshots now then > something else is happening in

Re: Backup scripts

2014-09-04 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Do you actually have any snapshots currently? From the scripts you posted it seems to just be rsyncing to the same dir every run and only claiming to be making snapshots. If you do have snapshots now then something else is happening in addition to th

Re: Backup scripts

2014-09-04 Thread Chris
On Thu, 2014-09-04 at 12:46 -0400, Kevin Korb wrote: > The scripts you posted look the same to me. And I don't see any form > of snapshotting. The $NOW variable is set and is echoed but it is > never actually used. For an rsync snapshot I would expect to see > either rsync --link-dest or a cp -a

Re: Backup scripts

2014-09-04 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The scripts you posted look the same to me. And I don't see any form of snapshotting. The $NOW variable is set and is echoed but it is never actually used. For an rsync snapshot I would expect to see either rsync --link-dest or a cp -al depending on

Backup scripts

2014-09-04 Thread Chris
I have two scripts that a kind soul on this list wrote for me over 4yrs ago. I got to looking at them the other day because my old box crashed and had to build a new one also got a new backup USB drive since I'm still copying over things from the old one. The first one is for a full backup: http:/