Hi, Another approach, instead of using volume-per-job, would be to allow multiple jobs per volume but with a volume use duration, and have run-after-job add the volume to a list of volumes to be replicated, that list acting as a queue for a separate tool which dequeues and replicates each volume ONLY when it has been marked USED.
How to achieve this without performance issues. ----- Original Message ----- From: "Phil Stracchino" <ph...@caerllewys.net> To: "bacula-users" <bacula-users@lists.sourceforge.net> Sent: Wednesday, August 23, 2017 12:12:43 AM Subject: Re: [Bacula-users] Bacula driver misconceptions On 08/22/17 14:27, Dimitri Maziuk wrote: > On 08/22/2017 12:27 PM, Phil Stracchino wrote: > >>> So I've a post-backup script that figures out which of the vchanger >>> "magazines" is the "current" one and rsyncs it to anther drive. That way >>> if a drive dies during write, I lose at most the latest backup. >> >> That sounds like a pretty good idea. After-the-fact redundancy. > > It isn't: what you want is to write to two disks at once. But if the > software wasn't designed for that, that's problems all the way to the > catalog. Oh sure, simultaneous two-volume write would arguably be the best solution. (Though there are viewpoints from which it's not. What if the job fails at 90% complete? You've wasted all that duplicate writing.) However, making a redundant copy of the volume in the background *as soon as the job completes* is not a bad next-best, and could arguably actually be more efficient if the copy is fired off in the background as a run-after-job if AND ONLY if the job successfully completes. > The other problem is that rsyncing an umpteen-TB drive can take a > non-trivial amount of time and ram. I would argue that if you're doing a full checksum rsync of a multi-terabyte drive, you're probably going about the task wrong. If I were to implement a scheme like this, I'd do volume-per-job, I'd use a run-after-job that appended the volume name to a list of volumes to be copied, and I wouldn't use rsync at all. When you already know you want to copy an entire volume, rsync is an inefficient way to do it. Another approach, instead of using volume-per-job, would be to allow multiple jobs per volume but with a volume use duration, and have run-after-job add the volume to a list of volumes to be replicated, that list acting as a queue for a separate tool which dequeues and replicates each volume ONLY when it has been marked USED. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Muhasin C.M Systems Engineer R & D Assistanz Networks Pvt LTD ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
