[Bacula-users] backuppc pool backup slow
Hello, I am experiencing slow backup performance, especially when backing up my BackupPC pool to tape. My system: Dual Opteron 2.4Ghz 1GB Ram, 4*300GB SATA (3Ware Controller RAID 5), Adaptec U160 Gentoo Linux (stage 1 install) kernel: linux-2.6.11-gentoo-r6 BackupPC version 2.1.1, pool resides on LVM2 volume Bacula version 1.36.3, MySQL store HP Ultrium-1 LTO Tape Exabyte VXA2 Autochanger To the LTO drive, I have seen as high as 12 Mb/s doing backups of other volumes. Backups of the backupPC pool are made from a snapshot, which is created by a 'RunBeforeJob' script. The last backup I ran (55GB of data) initially ran at around 2-3Mb/sec, which is not great, but as long as the job gets done within 4-6 hours I can deal. What seems peculiar is that when the total bytes reaches 55GB (should be everything) the job continues to run, for a long, long time. The bytes does not change but the files examined value changes. See my current job: Director connected at: 17-May-05 08:50 JobId 3 Job BackupPC.2005-05-16_13.17.16 is running. Backup Job started: 16-May-05 13:17 Files=2,326,531 Bytes=55,852,725,733 Bytes/sec=793,734 Files Examined=2,744,098 Processing file: /mnt/snapshot/backuppc/cpool/8/9/b/89ba699a9f1dde5a324a5f14ef626b77 SDReadSeqNo=5 fd=7 So far a whopping 16 HOURS and still going! What's going on? I would like to get my tape backups working correctly so I can start sleeping well again... - Jeff --- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the first software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=7412&alloc_id=16344&op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] backuppc pool backup slow
I would like to gain some understanding as to what bacula is doing when the bytes written stops changing and files examined keeps increasing. Is this a cataloging operation? Backuppc supports some kind of archiving feature, I wonder if anyone has used this in conjunction with bacula... Thanks, Jeff --- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the first software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=7412&alloc_id=16344&op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Problems getting two tape drive strategy working
Hello, I have a backup server, which acts as a repository for network backups. Attached to this server is a HP Ultrium-1 LTO drive and an Exabyte VXA2 Autoloader drive. What I am trying to do is have Monthly full backups go to the LTO drive, and the daily incremental backups go to the Exabyte. Weekly differential backups will go to the Exabyte initially, but I may decide later to send these to the LTO. I would like to split the backups this way so I have minimal tape changes, and the LTO tapes can go into our safe when they fill up, for long term archival. I am having trouble getting this to work in bacula: I created two pools, one for full backups and one for incremental and differential. Only media for the LTO is added to the full backup pool, as only media from the exabyte is added to the incremental pool. One problem that I have is that a backup will get upgraded from incremental to full (due to changes in my retention periods after testing), and bacula wants to use media from the incremental pool AND the LTO tape. I do not know why it will not use the other media pool... Maybe someone can offer me some suggestions to achieve by design goal, ideally I would have minimal intervention with the backup system, archiving the LTO tapes as neccessary and letting the VXA tapes recycle... Thanks Jeff Ramen --- This SF.Net email is sponsored by Yahoo. Introducing Yahoo! Search Developer Network - Create apps using Yahoo! Search APIs Find out how you can build Yahoo! directly into your own Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Raw LVM2 partition backup
Hello, Sorry, in my previous post I forgot to include the bacula output for the job: 10-Jun 09:21 baksvr-dir: Created new FileSet record "mylvmpartition_raw" 2005-06-10 09:21:02 10-Jun 09:21 baksvr-dir: Start Backup JobId 113, Job=BackupPC.2005-06-10_09.20.43 10-Jun 09:21 baksvr-sd: Volume "FULL_051605" previously written, moving to end of data. 10-Jun 09:23 baksvr-sd: Ready to append to end of Volume "FULL_051605" at file=59. 10-Jun 09:23 baksvr-sd: Alert: cannot open SCSI device '*None*' - No such file or directory 10-Jun 09:23 baksvr-dir: Bacula 1.36.3 (22Apr05): 10-Jun-2005 09:23:38 JobId: 113 Job:job.2005-06-10_09.20.43 Backup Level: Full Client: baksvr-fd FileSet:"mylvmpartition_raw" 2005-06-10 09:21:02 Pool: "MonthlyFullBackups" Storage:"HPUltrium" Start time: 10-Jun-2005 09:20:45 End time: 10-Jun-2005 09:23:38 FD Files Written: 1 SD Files Written: 1 FD Bytes Written: 0 SD Bytes Written: 101 Rate: 0.0 KB/s Software Compression: None Volume name(s): FULL_051605 Volume Session Id: 1 Volume Session Time:1118409615 Last Volume Bytes: 55,272,840,563 Non-fatal FD errors:0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination:Backup OK 10-Jun 09:23 baksvr-dir: Begin pruning Jobs. 10-Jun 09:23 baksvr-dir: No Jobs found to prune. 10-Jun 09:23 baksvr-dir: Begin pruning Files. 10-Jun 09:23 baksvr-dir: No Files found to prune. 10-Jun 09:23 baksvr-dir: End auto prune. Thanks, Jeff Ramen --- This SF.Net email is sponsored by: NEC IT Guy Games. How far can you shotput a projector? How fast can you ride your desk chair down the office luge track? If you want to score the big prize, get to know the little guy. Play to win an NEC 61" plasma display: http://www.necitguy.com/?r=20 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Raw LVM2 partition backup
Hello,
I cannot seem to make a raw (image) backup of a LVM2 partition:
(from bacula-dir.conf)
FileSet {
Name = mylvmpartition_raw
Include {
Options {
sparse = yes
}
File = /dev/vg/mylvmpartition
}
}
(my system):
Dual Opteron 2.4Ghz 1GB Ram, 4*300GB SATA (3Ware Controller RAID 5),
Adaptec U160
Gentoo Linux (stage 1 install) kernel: linux-2.6.11-gentoo-r6
Bacula version 1.36.3, MySQL store
HP Ultrium-1 LTO Tape
Exabyte VXA2 Autochanger
Thanks,
Jeff Ramen
---
This SF.Net email is sponsored by: NEC IT Guy Games. How far can you shotput
a projector? How fast can you ride your desk chair down the office luge track?
If you want to score the big prize, get to know the little guy.
Play to win an NEC 61" plasma display: http://www.necitguy.com/?r=20
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Raw LVM2 partition backup
I have worked around my problem with raw LVM2 partition backups.
FYI, I am doing this to backup my BackupPC pool, which resides entirely
on a LVM2 partition. This is
substantially faster then trying to do a file backup of the BackupPC
pool, due to the hard link handling
in bacula. The space requirement on tape does not change much as the
BackupPC pool is already
compressed.
I found that I had to use a FIFO to perform the raw backup. For
whatever reasonm I just could not get bacula
to write the raw partition using File = /dev/vg/partition in my fileset
directive.
Since I could not find any documentation for setting up a fifo from a
runBeforeJob script, I began experimenting...
What I found was that I could not just write a script that would create
a FIFO and then background a 'dd' command, bacula would hang on the 'dd'
process.
My solution was to write a little (very basic) perl server (this script,
of course, will be further developed):
#! /usr/bin/perl
my $fifo = '/tmp/cmdfifo';
while (1) {
my $cmd = '';
open(FIFO, $fifo);
while () {
$cmd = $_;
}
close FIFO;
print "fifo commander: received command '$cmd'\n";
if ($cmd =~ /^BACKUP/) {
print "fifo commander: executing backup setup script...\n";
system('/etc/bacula/backuppc_create_snapshot');
}
}
exit;
(end script)
(The fifo's in the /tmp directory were created before execution)
In my bacula-dir.conf job definition I just use the line:
RunBeforeJob = "sh -c 'echo BACKUP >>/tmp/cmdfifo'"
Which sends a command to my script, which runs the script in /etc/bacula
to create a snapshot, and start
'dd':
#! /bin/sh
#
# backuppc_create_snapshot
#
# Bash script to unmount old snapshot, then create new one. Designed to
be called by bacula
# before backups written.
#
#umount /mnt/snapshot/backuppc
lvremove -f /dev/vg/snap_backuppc
lvcreate -s -n snap_backuppc -L50G /dev/vg/backuppc2
#mount /dev/vg/snap_backuppc /mnt/snapshot/backuppc
setsid dd if=/dev/vg/snap_backuppc of=/tmp/testfifo bs=16384
(end script)
This system worked. I was able to back up my BackupPC pool in about
4hrs as opposed to 20+ it took before.
Maybe this is not the best way of doing things, but it works and seems
to perform well. I am thinking of enchancing my perl server to take
more complex command to set up any snapshot/partition...
Jeff
---
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
