Bill,
Thank you so much for the response. This is the best possible answer I could
have received. I will put your script in play. I will most likely make
changes to have this run automagically š say quarterly. Modifications would be
to pull jobs ids for those jobs and feed the script. Unless I can somehow
figure out a way with the schedule Job-overrides to assign a RunScript. Say:
(I know pie in the sky)
Schedule {
Name = "My_Cycle"
Run = Level=Full feb mar may jun aug sep nov dec 1st - 5th sat at 00:00
Run = Level=Full RunScript = AutoRestoreAndVerify jan apr jul oct 1st - 5th
sat at 00:00
Run = Level=Differential sun at 18:00
Run = Level=Incremental mon at 18:00
Run = Level=Differential tue at 18:00
Run = Level=Incremental wed at 18:00
Run = Level=Differential thu at 18:00
Run = Level=Incremental fri at 18:00
}
Thanks again I will update the group with my experience.
Jim Richardson
-----Original Message-----
From: Bill Arlofski [mailto:waa-bac...@revpol.com]
Sent: Thursday, August 31, 2017 10:30 PM
To: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Bacula Verification Procedures
On 08/30/2017 09:31 PM, Jim Richardson wrote:
> So I am looking to the community to help with the following questions
> to save me from trial and error:
Hi Jim!
> 1. Is there a verification job or a sequence of jobs ābuilt inā to Bacula
> that can accomplish an automated restore with an automated verification of
> Volume to Restore on Disk?
No, but some of this can be scripted.
I have attached a script that I have been using here for some months, both on
my Bacula Community environment and in my Bacula Enterprise test environments.
The idea is that in a Runscript (RunsWhen = after), this script is called which
then does an automated restore of the current backup job, then runs
VolumeToCatalog, DiskToCatalog, and Data verification jobs. I am sure it is not
perfect yet, but I bet this can provide some ideas and a jumping point for you.
Of course, since it accepts a jobid, client, fileset, and jobname on the
command line, you can run the restore, followed by the verifies on any job you
like and it does not have to be called from a Job's RunScript stanza. (Random
Job restore and verification tests, anyone?)
> 2. Will the VolumeToCatalog job verify Catalog files to the Tape without
> first hashing and when hardware compression is in place?
This one is "above my pay grade"ā¢ :)
Actually, with hardware compression, that is completely transparent to Bacula.
The hashing is computed, the data is written to the drive, the drive does its
hardware compression, and writes the compressed data to the tape. On a
read/verify, the opposite takes place, and Bacula sees the actual
(uncompressed) data, and the hashes will match.
> 3. Is there a verification job or a sequence of jobs ābuilt inā to Bacula
> that can accomplish an automated Restore with an automated Verification of
> Tape to Restore on Disk?
This is the same as #1, just with "tape" swapped for "volume". In my mind, file
volumes and tape volumes can be considered and treated the same in most
scenarios.
> 4. Are there other options or best practices to accomplish both a post job
> verification of Catalog to Volume and a periodic verification of Volume
> (Tape or Disk) to Restore on Disk?
Nothing that I know of. I suspect that since (most of) the pieces are there:
RunScript{} stanzas, restore and verify jobs, custom scripts, etc. I think that
the only "best practice" would be what you or your company requires. eg:
How often should restores take place, which jobs/volumes should be tested,
should random jobs be tested, or should only some critical ones be
automatically tested, etc...
> 5. Is there a Bacula Systems whitepaper that covers best practices for
> Verification and Testing of backups?
For this, I can answer a definite "no". I think it is mainly because it kind of
falls under #4, where a company's policy would probably dictate such specific
things.
> I have looked into DiskToCatalog this will not work. (someone argue with me).
Nope... :) I will only agree, and then add to your comment. :)
The issue you describe below is exactly the issue I plan on reporting
internally as a feature request to Bacula Systems. My idea is exactly the one
you describe below, and I cannot see a way that this can be accomplished as
Bacula is currently written.
> It will compare files on the client disk to the catalog. After 15
> hours + verification run time many files have changes - archived,
> reached retention period and deleted, etc. Ideally, I want a complete
> restore to different location and a verification that the files in the
> Catalog and the Volume are actually the files on disk after the Restore.
I think (and I will let Kern answer for himself of course :), but I think Kern
will tell you that if the backup Job has a "Signature = md5" (or sha1), and it
ran with no errors, and then the restore job worked and reported no errors,
then for all intents and purposes, the restore is 100% fine, your data that was
restored is exactly what was backed up on the volumes and no further tests are
necessary.
Kern?? :)
While I completely agree with that logic, I do also like the idea of being able
to ask Bacula to do a verify job against data that was restored to a different
directory after the restore has finished. As a matter of fact, when running
tests last year to certify Bacula Enterprise Edition with HPE's "StoreOnce"ā¢
storage devices, they required "diff" tests to verify that the data that was
restored was the same as the static data in the test directories we were
backing up. They would not accept my sound reasoning above, so I think there
can be some argument for such a feature to prove beyond "trust us that we
verify the data on the way in and on the way out..."
Please see my attached script (AutoRestoreAndVerify.sh), and let me know if
this is useful. I plan to post it to my website
(http://www.revpol.com/bacula/) at some point when I have a bit of time. And,
if a feature is adopted and implemented to allow a Verify job to verify data
that was restored to a different location, I will revisit this script to make
sure that is considered as one of the tests.
Best regards,
Bill
--
Bill Arlofski
http://www.revpol.com/bacula
-- Not responsible for anything below this line --
CONFIDENTIALITY: This email (including any attachments) may contain
confidential, proprietary and privileged information, and unauthorized
disclosure or use is prohibited. If you received this email in error, please
notify the sender and delete this email from your system. Thank you.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
