Re: [Bacula-users] VirtualFull using tape drives

Thu, 06 Jan 2011 19:34:56 -0800

On 7/01/2011 3:31 AM, Jesper Krogh wrote: 
Hi

Anyone doing VirtualFull backups using tapedrives only?
Can they shortly describe their setup? pro/con/etc?

Thanks.

Yes I am.
I am using a TL2000 tape library with two drives.
The technique can't work if you only have one tape drive.

I'm taking incremental backups Mon-Fri.
Then after the incremental backups are finished on Friday I consolidate them into a VirtualFull backup.

For a VirtualFull backup to work it takes the previous full backup and the incremental backups since, and combines them to produce new tape(s) that will be promoted to being the latest full backups.
The first incremental backups ever run for a host will auto upgrade to a full backup so you're covered for that.

The process seems to work well for me, after I worked around a minor problem.
After the incremental backup is complete, tapes are left in the drives.
If tape X is in drive 1 after the incremental backups are complete and the VirtualFull wishes to load it it into drive 0 to read from; then Bacula can't eject the tape from drive 1 and then load it into drive 0 and deadlocks for user intervention.
User intervention doesn't help either as the tape you want to eject is locked by Bacula, so you end up stopping daemons and interrupting the backup.

The solution is to use an administrative job that is scheduled to eject all tapes from the drives after the incremental backups are done, and before the VirtualFull backup starts.
Perhaps I should have just raised a bug on that...

In your Incremental Backup Job definitions make sure you set Accurate = yes

Also the Director must have 'Maximum Concurrent Jobs' set to at least 2 in order for the VirtualFull backups to work (one reader and one writer I guess).

I have created a /etc/bacula/scripts/release_tapes script with the following contents to eject the tapes from the drives:
#!/bin/bash

/usr/bin/bconsole <<EOF
release drive=0
release drive=1
quit
EOF
These are the relevant entries in my /etc/bacula/bacula-dir.conf file:
# This job definition contains all compulsory job values even if they aren't used by the job type.
#
JobDefs {
  Name = "DefaultJob"
  Client = bacula-fd
  FileSet = "LinuxSet"
  Messages = Standard
  Pool = Default
}

# Default values to be included in backup jobs.
#
JobDefs {
  Name = "BackupJob"
  Type = Backup
  Level = Incremental
  Client = bacula-fd
  FileSet = "LinuxSet"
  Schedule = "DailyBackupSchedule"
  Messages = Standard
  Pool = Default
  # If an incremental backup gets upgraded to a Full backup, then send its output to FullPool.
  Full Backup Pool = FullPool
  # Bootstrap file will be named after the job.
  Write Bootstrap = "/var/lib/bacula/%n.bsr"
  # 'Accurate = yes' will detect files that have been moved but still have old time stamps.
  # Doing this check will use up a lot more memory on the client but is necessary for a permanent incremental strategy to work.
  Accurate = yes
}

# An admin job to release tapes.
#
Job {
  Name = "ReleaseTapes"
  JobDefs = "DefaultJob"
  Type = Admin
  Schedule = "ReleaseTapesSchedule"
  RunAfterJob = "/etc/bacula/scripts/release_tapes"
}

# Incremental backups on the week days.
# Spool the incremental backups to disk to prevent tape shoe-shine.
# Consolidate the incremental backups into a full backup on Friday.
# Set the priority of the VirtualFull backups to happen after Incremental backups are complete and the tapes are released,
# but before the catalog backup happens.
Schedule {
  Name = "DailyBackupSchedule"
  Run = Level=Incremental Priority=11 SpoolData=yes mon-fri at 22:05
  Run = Level=VirtualFull Priority=13               fri     at 22:15
}

# Release the tapes from the drives on Friday after the Incremental are finished, but before the VirtualFulls start.
# This is to work around a problem where sometimes the VirtualFull requests to load a tape in a drive that is already in the other and it gets stuck.
#
Schedule {
  Name = "ReleaseTapesSchedule"
  Run = Priority=12 fri at 22:10
}

# Default pool definition used by incremental backups.
# We wish to be able to restore files for any day for at least 2 weeks, so set the retention to 13 days.
#
Pool {
  Name = Default
  Volume Retention = 13 days
  Pool Type = Backup
  # Automatically prune and recycle volumes.
  AutoPrune = yes
  Recycle = yes
  # Do not use tapes who's labels start with CLN since they are cleaning tapes.
  Cleaning Prefix = "CLN"
  Storage = TL2000
  # Get tapes from scratch pool and return them to the scratch pool when they are purged.
  Scratch Pool = Scratch
  Recycle Pool = Scratch
  # The location where the VirtualFull backups will be written to.
  Next Pool = FullPool
}

# Pool used by Full and VirtualFull backups.
# Keep for 3 weeks, so set the retention to 20 days. (3*7-1)
Pool {
  Name = FullPool
  Volume Retention = 20 days
  Pool Type = Backup
  # Automatically prune and recycle volumes.
  AutoPrune = yes
  Recycle = yes
  # Do not use tapes whos labels start with CLN since they are cleaning tapes.
  Cleaning Prefix = "CLN"
  Storage = TL2000
  # Get tapes from scratch pool and return them to the scratch pool when they are purged.
  Scratch Pool = Scratch
  Recycle Pool = Scratch
  # The location where the copies go for offsite backups.
  Next Pool = CopyPool
}

In the two schedule definitions you can see that the incremental backups are set at priority 11, then the admin job to eject the tape is at priority 12, and then the VirtualFull is at priority 13.
This all makes sure that they happen in the correct sequence.
I'd imagine that for this to work you shouldn't modify the settings that lets jobs with different priorities run at the same time.

In the last pool you can see a hint of me using Copy jobs as well.
I haven't included the entries for these, but after the VirtualFull backups are complete, I produce Copies of them for off-site rotation.
There were a few tricks related to them as well.
Mainly to do with the job not being allowed to run a SQL query to gather the list of latest full backups at the time it is scheduled as opposed to when it can run.
I solved this with an admin job to kick off the Copy job rather than scheduling it directly.

Regards,
----------
Jim Barber
DDI Health

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to 
best implement a security strategy that keeps consumers' information secure 
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl 
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Reply via email to