In the message dated: Thu, 27 Jul 2006 12:31:29 CDT,
The pithy ruminations from "Orman, David L [CNDE]" on
<RE: [Bacula-users] backup size (and estimate) much greater than actualdisk usa
ge> were:
=> Your problem is probably a sparse file. Specifically look at
=> /var/log/lastlog
=>
=> That gets ridiculously long on 64 bit systems. Heres mine:
=>
=> # ls -l lastlog
=> -r-------- 1 root root 1254130450140 Jul 27 12:28 lastlog
=> #
=>
=>
=> Solutions are to turn on sparse file support or exclude that file.
Bingo!
That's it, exactly.
I figured it was probably a sparse files issue, or somehow bacula was reading
the Linux Volume Manager devices and/or the EMC Powerpath devices in addition
to the mounted filesystems.
What was perplexing was that I was not seeing this problem on my other Linux
x86_64 systems.
Here's what I found:
CentOS 4.3 final, on x86_64 (with all the 64-bit stuff), has a
smaller /var/log/lastlog (7329492 bytes =~ 6.98MB), so the fact
that it's a sparse file isn't a problem.
RHEL4, on x86_64 (with all the 64-bit stuff), has a
larger /var/log/lastlog (1254130450140 bytes =~ 1.14TB).
Thanks,
Mark
=>
=> -----Original Message-----
=> From: [EMAIL PROTECTED]
=> [mailto:[EMAIL PROTECTED] On Behalf Of
=> [EMAIL PROTECTED]
=> Sent: Thursday, July 27, 2006 12:15 PM
=> To: bacula-users@lists.sourceforge.net
=> Subject: [Bacula-users] backup size (and estimate) much greater than
=> actualdisk usage
=>
=> I just added a new client (Linux, x86_64, RHEL4, 4GB RAM, bacula-fd
=> 1.38.9) to my existing bacula config. Backups of all other clients are
=> working fine.
=>
=> The new client has ~4.5TB of local disk, with about 6.5GB used (that'll
=> change fast!).
=>
=> The problem is that bacula "estimate" command reports that there's about
=> 1.14TB of data to be backed up. Attempting to backup this client seems
=> to confirm that bacula is trying to backup more data than actually
=> exists--it kept spooling data to the bacula-sd daemon until the job was
=> canceled (after more than 31GB was spooled).
=>
=> The bacula fileset and excludes, defined in bacula-dir.conf, are the
=> same for all my backup clients. That configuration should be backing up
=> all local filesystems (minus some specified files and directories).
=>
=>
=> Any ideas why bacula is trying to backup 1.14TB, when only ~6.5GB is
=> actually used, or how to get more info? Running "estimate" with "debug"
=> set to 200 doesn't produce more details.
=>
=> I'm also very interested in getting an idea what kind of overhead is
=> involved (client and server) with the exclude directives that I'm using,
=> particularly the "wildDir" and "wildFile" statements. Is there any way
=> to optimize the excludes, by ordering them differently or by ommitting
=> the trailing "*"?
=>
=>
=>
=> Here are some details:
=>
=> ------------------------- client disk config
=> ---------------------------- [EMAIL PROTECTED] /]# df -h
=> Filesystem Size Used Avail Use% Mounted on
=> /dev/sda1 9.9G 3.8G 5.6G 41% /
=> none 2.0G 0 2.0G 0% /dev/shm
=> /dev/sda3 57G 2.6G 51G 5%
=> /state/partition1
=> tmpfs 965M 5.0M 960M 1%
=> /var/lib/ganglia/rrds
=> /dev/mapper/data1vg-sbiahome_lv 493G 102M 493G 1% /sbia/home
=> /dev/mapper/data1vg-sbiaproj_lv 2.0T 103M 2.0T 1%
=> /sbia/projects
=> /dev/mapper/data1vg-sbiacompspace_lv 2.0T 103M 2.0T 1%
=> /sbia/comp_space
=>
=> [EMAIL PROTECTED] /]# mount
=> /dev/sda1 on / type ext3 (rw)
=> none on /proc type proc (rw)
=> none on /sys type sysfs (rw)
=> none on /dev/pts type devpts (rw,gid=5,mode=620) usbfs on /proc/bus/usb
=> type usbfs (rw) none on /dev/shm type tmpfs (rw)
=> /dev/sda3 on /state/partition1 type ext3 (rw) tmpfs on
=> /var/lib/ganglia/rrds type tmpfs (rw,size=1011256000,gid=99,uid=99)
=> none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on
=> /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
=> automount(pid2804) on /home type autofs
=> (rw,fd=5,pgrp=2804,minproto=2,maxproto=4)
=> nfsd on /proc/fs/nfsd type nfsd (rw)
=> /dev/mapper/data1vg-sbiahome_lv on /sbia/home type ext3 (rw)
=> /dev/mapper/data1vg-sbiaproj_lv on /sbia/projects type ext3 (rw)
=> /dev/mapper/data1vg-sbiacompspace_lv on /sbia/comp_space type ext3 (rw)
=> ------------------------------------------------------------------------
=> -
=>
=>
=>
=> -------------excerpts from bacula-dir.conf
=> ------------------------------
=>
=> #
=> # Default Bacula Director Configuration file # # For Bacula release
=> 1.38.9 (02 May 2006) -- redhat (Bordeaux) #
=>
=> JobDefs {
=> Name = "DefaultJob"
=> Type = Backup
=> Level = Incremental
=> FileSet = "Full Set"
=> Messages = Standard
=> Priority = 10
=> Storage = pv132t
=> Prefer Mounted Volumes = no # Try to use both tape drives at once
=> Maximum Concurrent Jobs = 1 # so that only one instance of each job
=> is running
=> }
=>
=> Job {
=> Name = "olympus-inc"
=> Client = olympus-fd
=> JobDefs = "DefaultJob"
=> Write Bootstrap = "/usr/local/bacula/var/working/olympus.bsr"
=> SpoolData = yes
=> Schedule = "Incremental-Mon"
=> Pool = "Incremental"
=> }
=> Job {
=> Name = "olympus-full"
=> Client = olympus-fd
=> JobDefs = "DefaultJob"
=> Write Bootstrap = "/usr/local/bacula/var/working/olympus.bsr"
=> SpoolData = yes
=> Schedule = "Full-Mon"
=> Pool = "Full"
=> }
=>
=>
=> # List of files to be backed up
=> FileSet
=> {
=> Name = "Full Set"
=> Include
=> {
=> Options
=> {
=> fstype = ext2
=> fstype = xfs
=> fstype = ufs
=> onefs = no
=> signature = MD5
=> Exclude = yes
=> wildFile = "swapfile*"
=> wildFile = "swap*.*gb"
=> wildDir = "/var/cache/*"
=> wildDir = "/var/run/*"
=> wildDir = "/var/spool/clientmqueue/*"
=> wildDir = "/var/spool/cups/tmp/*"
=> wildDir = "/lost+found/*"
=> wildDir = "*/comp_space/*"
=> wildDir = "/dev/mapper/*"
=> wildDir = "/dev/pts/*"
=> wildDir = "/dev/shm/*"
=> wildDir = "/media/*"
=> wildDir = "/proc/*"
=> wildDir = "/sys/*"
=> wildDir = "/tmp/*"
=> wildDir = "/var/lock/*"
=> wildDir = "/var/spool/bacula/*"
=> wildDir = "/var/tmp/*"
=> }
=> File = /
=> }
=>
=>
=> # Exclude Stuff
=> Exclude
=> {
=> File = core
=> File = /.fsck
=> File = /.journal
=> }
=> }
=>
=> Client {
=> Name = olympus-fd
=> Address = olympus
=> FDPort = 9102
=> Catalog = MyCatalog
=> Password = "foobar"
=> File Retention = 30 days # 30 days
=> Job Retention = 6 months # six months
=> AutoPrune = yes # Prune expired Jobs/Files
=> Maximum Concurrent Jobs = 20
=> }
=>
=> ------------------------------------------------------------------------
=> -
=>
=>
=> ----
=> Mark Bergman [EMAIL PROTECTED]
=> System Administrator
=> Section of Biomedical Image Analysis 215-662-7310
=> Department of Radiology, University of Pennsylvania
=>
=> http://pgpkeys.pca.dfn.de:11371/pks/lookup?search=mark.bergman%40.uphs.u
=> penn.edu
=>
=>
=>
=>
=> The information contained in this e-mail message is intended only for
=> the personal and confidential use of the recipient(s) named above. If
=> the reader of this message is not the intended recipient or an agent
=> responsible for delivering it to the intended recipient, you are hereby
=> notified that you have received this document in error and that any
=> review, dissemination, distribution, or copying of this message is
=> strictly prohibited. If you have received this communication in error,
=> please notify us immediately by e-mail, and delete the original message.
=>
=> ------------------------------------------------------------------------
=> -
=> Take Surveys. Earn Cash. Influence the Future of IT Join
=> SourceForge.net's Techsay panel and you'll get the chance to share your
=> opinions on IT & business topics through brief surveys -- and earn cash
=> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE
=> V
=> _______________________________________________
=> Bacula-users mailing list
=> Bacula-users@lists.sourceforge.net
=> https://lists.sourceforge.net/lists/listinfo/bacula-users
=>
The information contained in this e-mail message is intended only for the
personal and confidential use of the recipient(s) named above. If the reader of
this message is not the intended recipient or an agent responsible for
delivering it to the intended recipient, you are hereby notified that you have
received this document in error and that any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you have
received this communication in error, please notify us immediately by e-mail,
and delete the original message.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
