On 01/15/10 08:48, John Drescher wrote:
> I think others were looking for the more complicated task of deleting
> file volumes automatically after their retention period had expired.
Having now finished the OS reinstallation on my main workstation, I've
had time to look at this problem. Here's my working example.
To start with, let's take a look at my Pool setup:
Pool {
Name = Scratch
Storage = babylon4-sd
Pool Type = Backup
}
Pool {
Name = Full-Disk
Storage = babylon4-sd
Pool Type = Backup
Recycle = no
Recycle Oldest Volume = no
Recycle Current Volume = no
AutoPrune = yes # Prune expired volumes
Volume Retention = 2 months
Maximum Volume Jobs = 0
Volume Use Duration = 23h
Label Format =
"FULL-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
RecyclePool = Scratch
}
Pool {
Name = Diff-Disk
Storage = babylon4-sd
Pool Type = Backup
Recycle = no
Recycle Oldest Volume = no
Recycle Current Volume = no
AutoPrune = yes
Volume Retention = 2 months
Maximum Volume Jobs = 0
Volume Use Duration = 23h
Label Format =
"DIFF-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
RecyclePool = Scratch
}
Pool {
Name = Incr-Disk
Storage = babylon4-sd
Pool Type = Backup
Recycle = no
Recycle Oldest Volume = no
Recycle Current Volume = no
AutoPrune = yes
Volume Retention = 1 month
Maximum Volume Jobs = 0
Volume Use Duration = 23h
Label Format =
"INCR-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
RecyclePool = Scratch
}
Pool {
Name = Full-Tape
Storage = babylon5-sd
Pool Type = Backup
Recycle = yes
Autoprune = yes
Volume Retention = 365d
Recycle Oldest Volume = yes
Recycle Current Volume = yes
Label Format = "ARCH-"
Maximum Volumes = 9
}
As you see, there are three separate Disk pools, divided by level, with
different retention periods, all of which recycle into the Scratch pool.
Each disk volume is used for a single day's backups, then marked used,
enabling good granularity of disk usage. There is also a separate tape
pool, volumes in which are recycled back into the same Pool ready for reuse.
Now, with that, on to the autoclean script:
#!/usr/bin/perl
use strict;
use Getopt::Long;
use IPC::Open2;
use IO::Handle;
my $bconsole = '/opt/bacula/bin/bconsole';
my (%opts, @purged, $pid);
GetOptions(\%opts,
'verbose|v',
'test');
my ($IN, $OUT) = (IO::Handle->new(), IO::Handle->new());
$pid = open2($OUT, $IN, $bconsole);
if (scalar (@purged = check_volumes()))
{
printf("Bacula reports the following purged volumes:\n\t%s\n",
join("\n\t", @purged)) if ($opts{verbose});
my $deleted = delete_volumes(@purged);
print "$deleted volumes deleted.\n" if ($opts{verbose});
}
elsif ($opts{verbose})
{
print "No purged volumes found to delete.\n";
}
print $IN "exit\n";
waitpid($pid, 0);
exit (0);
sub check_volumes
{
my $dividers = 0;
my (@purged, @row);
print $IN "list volumes pool=Scratch\n";
for (;;)
{
my $resp = <$OUT>;
last if ($resp =~ /No results to list./);
$dividers++ if ($resp =~ /^[\+\-]+$/);
last if ($dividers == 3);
@row = split(/\s+/, $resp);
push (@purged, $row[3]) if ($row[5] eq 'Purged');
}
return (@purged);
}
sub delete_volumes
{
my $volume_dir = '/spool/bacula/';
my $count = 0;
foreach my $vol (@_)
{
my $l;
my $file = $volume_dir.$vol;
print "Deleting volume $vol from catalog ... " if ($opts{verbose});
print $IN "delete volume=$vol yes\n";
$l = <$OUT>;
$l = <$OUT>;
print "Done.\nDeleting volume $file from disk ... " if
($opts{verbose});
if (-f $file)
{
$count++;
unlink ($file);
}
print "Done.\n" if ($opts{verbose});
}
return ($count);
}
This script leaves volume expiration entirely to Bacula. It asks Bacula
for a list of volumes that have been expired and moved into the Scratch
pool. If there are none, it returns immediately; otherwise, it
double-checks to make sure that each of those volumes has been purged,
builds a list of purged volumes, then deletes the purged volumes first
from the Catalog, then from disk.
I haven't yet set this up as an Admin job, but I have tested it
thoroughly and verified that it works, including properly exiting if the
Scratch pool is empty.
--
Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355
ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org
Renaissance Man, Unix ronin, Perl hacker, Free Stater
It's not the years, it's the mileage.
------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
