2009/1/8 Chris Hoogendyk <hoogen...@bio.umass.edu>:
>
>
> Alan Brown wrote:
>> On Wed, 7 Jan 2009, Chris Hoogendyk wrote:
>>
>>> Most backup software leaves the responsibility for planning more in the
>>> sysadmin's hands. If the software is just writing data to the tape until
>>> it hits the end of tape and then asking for another, hardware
>>> compression is a logical choice.
>>>
>>
>> Part of the preference for no hardware compression in Amanda and other
>> packages is that a lot of them assume fixed tape sizes, so the only
>> thing that hardware compression gets you is less time to write to that
>> maximum.
>>
>> ie: Program assumes tape is 33Gb. When 33Gb is reached the program
>> demands the tape be changed, even if the tape is only physically half
>> used thanks to hardware compression
>>
>> Most of these types of programs have an all-or-nothing approach to
>> restores. This is not exactly the bacula model.
>>
>> Even with hardware compression switched off, spooling is likely required
>> to prevent shoeshining of modern tape drives. Spooling hardware MUST be
>> fast enough to keep up. Switching on hardware compression increases those
>> requirements but the budgetary increase is minor.
>
>
> Most of your batch of replies were on the mark. This one is at least in
> part incorrect and confusing.
>
> From the linux man page for amanda.conf:
>
> "Once the backups start, Amanda will continue to write to a tape until
> it gets an error, regardless of what value is entered for [tape] length"
>
> In other words the length entered in the tapetype definition is only
> used for planning and scheduling. The "error" will normally be the end
> of tape "error", which allows for a variable amount of data actually
> being written to the tape, which would include allowing for variable
> hardware compression results.
>
> Also, I have no idea what you mean by "all or nothing restores". That's
> just wrong.
>
>
> --
> ---------------
>
> Chris Hoogendyk
>
> -
> O__ ---- Systems Administrator
> c/ /'_ --- Biology & Geology Departments
> (*) \(*) -- 140 Morrill Science Center
> ~~~~~~~~~~ - University of Massachusetts, Amherst
>
> <hoogen...@bio.umass.edu>
>
> ---------------
>
> Erdös 4
>
>
>
> ------------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It is the best place to buy or sell services for
> just about anything Open Source.
> http://p.sf.net/sfu/Xq1LFB
> _______________________________________________
Hi I really appreciate the discussion about advantages and
disadvantages of compression. But please bear in mind that I am not
using compression *in anyway*. So please could you help me to solve
this problem. Below I let you config files (a bit shortened and
changed in obvious parts like Password), JOb report and info from mt
and tapeinfo:
Some Clarifications: DirectorandStorageServerThatHasDat72, is the
Director and Storage Server That Has Dat 72 ;)
Bulldog, Clark are bacula clients
StorageServerThasHasLTO3: Storage Server Thas Has LTO3 ;)
Job Report:
08-Jan 16:40 DirectorandStorageServerThatHasDat72-dir-dir JobId 1975:
Bacula DirectorandStorageServerThatHasDat72-dir-dir 2.4.2 (26Jul08):
08-Jan-2009 16:40:04
Build OS: i686-redhat-linux-gnu redhat
JobId: 1975
Job: ProbandoLTO3otherclient.2009-01-08_15.37.08
Backup Level: Full (upgraded from Differential)
Client: "sinclair-fd" 2.4.3 (10Oct08)
x86_64-unknown-linux-gnu,redhat,
FileSet: "dumpProbandoLTO3otherclient" 2009-01-07 10:40:19
Pool: "ProbandoLTO3otherclient" (From Job resource)
Storage: "LTO3" (From Job resource)
Scheduled time: 08-Jan-2009 15:37:02
Start time: 08-Jan-2009 15:37:07
End time: 08-Jan-2009 16:40:04
Elapsed time: 1 hour 2 mins 57 secs
Priority: 11
FD Files Written: 490,018
SD Files Written: 490,018
FD Bytes Written: 11,616,880,555 (11.61 GB)
SD Bytes Written: 11,678,187,032 (11.67 GB)
Rate: 3075.7 KB/s
Software Compression: None
VSS: no
Storage Encryption: no
Volume name(s): TestWithSpoolingBig1
Volume Session Id: 1
Volume Session Time: 1231436187
Last Volume Bytes: 11,703,250,944 (11.70 GB)
Non-fatal FD errors: 2
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK -- with warnings
(Warnings are not a problem, are due files not found)
Config files from DirectorandStorageServerThatHasDat72:
bacula-dir.conf
Director {
Name = DirectorandStorageServerThatHasDat72-dir
DIRport = 9101 # where we listen for UA connections
QueryFile = "/etc/bacula/query.sql"
WorkingDirectory = "/var/bacula/working"
PidDirectory = "/etc/bacula"
Maximum Concurrent Jobs = 1
Password----
Messages = Daemon
}
Job {
Name = "BackupDogoTape001"
Type = Backup
Level = Differential
Client = dogo-fd
FileSet = "dumpDogo"
Schedule = "Sabado"
Storage = DAT72i
Pool = TapePoolSabado
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupDogo001.bsr"
Priority = 11
}
Job {
Name = "BackupDogoTape002"
Type = Backup
Level = Differential
Client = dogo-fd
FileSet = "dumpDogo"
Schedule = "Martes"
Storage = DAT72i
Pool = TapePoolMartes
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupDogo002.bsr"
Priority = 11
}
Job {
Name = "BackupDogoTape003"
Type = Backup
Level = Differential
Client = dogo-fd
FileSet = "dumpDogo"
Schedule = "Jueves"
Storage = DAT72i
Pool = TapePoolJueves
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupDogo003.bsr"
Priority = 11
}
Job {
Name = "BackupAcadTape004"
Type = Backup
Level = Differential
Client = acad-fd
FileSet = "dumpAcad"
Schedule = "Miercoles"
Storage = DAT72i
Pool = TapePoolAcadM
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupAcad004.bsr"
Priority = 11
}
Job {
Name = "BackupAcadTape005"
Type = Backup
Level = Differential
Client = acad-fd
FileSet = "dumpAcad"
Schedule = "Viernes"
Storage = DAT72i
Pool = TapePoolAcadV
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupAcad005.bsr"
Priority = 11
}
Job {
Name = "BackupTexasTape001"
Type = Backup
Level = Differential
Client = texas-fd
FileSet = "dumpTexas"
Schedule = "Sabado"
Storage = DAT72i
Pool = TapePoolSabado
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupTexas001.bsr"
Priority = 11
}
Job {
Name = "BackupTexasTape002"
Type = Backup
Level = Differential
Client = texas-fd
FileSet = "dumpTexas"
Schedule = "Martes"
Storage = DAT72i
Pool = TapePoolMartes
Messages = Standard
Write Bootstrap = "/var/bacula/working/BackupTexas002.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape1-1"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail001"
Schedule = "Sabado"
Storage = LTO3
Pool = TapePoolSabadoLTO3
Messages = Standard
SpoolData = yes # Agregador por Sergio: Bacula allows you to specify
that you want the Storage daemon to initially write your data to disk
and then subsequently to tape
#Maximum Spool Size = 3000000000
#Spool Directory = /var/spool/bacula
Write Bootstrap = "/var/bacula/working/BackupSinclairTape11.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape1-2"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail002"
Schedule = "Sabado"
Storage = LTO3
Pool = TapePoolSabadoLTO3
Messages = Standard
SpoolData = yes # Agregador por Sergio: Bacula allows you to specify
that you want the Storage daemon to initially write your data to disk
and then subsequently to tape
Write Bootstrap = "/var/bacula/working/BackupSinclairTape12.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape1-3"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail003"
Schedule = "Sabado"
Storage = LTO3
Pool = TapePoolSabadoLTO3
Messages = Standard
SpoolData = yes # Agregador por Sergio: Bacula allows you to specify
that you want the Storage daemon to initially write your data to disk
and then subsequently to tape
Write Bootstrap = "/var/bacula/working/BackupSinclairTape13.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape1-4"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail004"
Schedule = "Sabado"
Storage = LTO3
Pool = TapePoolSabadoLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape14.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape2-1"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail001"
Schedule = "Martes"
Storage = LTO3
Pool = TapePoolMartesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape21.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape2-2"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail002"
Schedule = "Martes"
Storage = LTO3
Pool = TapePoolMartesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape22.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape2-3"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail003"
Schedule = "Martes"
Storage = LTO3
Pool = TapePoolMartesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape23.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape2-4"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail004"
Schedule = "Martes"
Storage = LTO3
Pool = TapePoolMartesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape24.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape3-1"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail001"
Schedule = "Jueves"
Storage = LTO3
Pool = TapePoolJuevesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape31.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape3-2"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail002"
Schedule = "Jueves"
Storage = LTO3
Pool = TapePoolJuevesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape32.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape3-3"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail003"
Schedule = "Jueves"
Storage = LTO3
Pool = TapePoolJuevesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape33.bsr"
Priority = 11
}
Job {
Name = "SinclairMailTape3-4"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpSinclairMail004"
Schedule = "Jueves"
Storage = LTO3
Pool = TapePoolJuevesLTO3
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupSinclairTape34.bsr"
Priority = 11
}
Job {
Name = "BackupNewton"
Type = Backup
Level = Differential
Client = newton-fd
FileSet = "dumpNewton"
Storage = DAT72i
Pool = TapePool
Messages = Standard
Write Bootstrap = "c:\bacula\working\BackupNewton.bsr"
Priority = 11
}
Job {
Name = "ProbandoLTO3otherclient"
Type = Backup
Level = Differential
Client = sinclair-fd
FileSet = "dumpProbandoLTO3otherclient"
Schedule = "Jueves"
Storage = LTO3
Pool = ProbandoLTO3otherclient
Messages = Standard
SpoolData = yes
Write Bootstrap = "/var/bacula/working/BackupProbandoLTO3otherclient001.bsr"
Priority = 11
}
Job {
Name = "RestoreAcad"
Type = Restore
Client = acad-fd
FileSet = "dumpAcad"
Pool = TapePool
Storage = DAT72i
Messages = Standard
Where = /extra/backup/restore
}
Job {
Name = "RestoreDogo"
Type = Restore
Client = dogo-fd
FileSet = "dumpDogo"
Pool = TapePool
Storage = DAT72i
Messages = Standard
Where = /extra/backup/restore
}
Job {
Name = "RestoreSinclairMail001"
Type = Restore
Client = sinclair-fd
FileSet = "dumpSinclairMail001"
Pool = TapePool
Storage = LTO3
Messages = Standard
Where = /extra/backup/restore
}
Job {
Name = "RestoreSinclairMail002"
Type = Restore
Client = sinclair-fd
FileSet = "dumpSinclairMail002"
Pool = TapePool
Storage = LTO3
Messages = Standard
Where = /extra/backup/restore
}
Job {
Name = "RestoreSinclairMail003"
Type = Restore
Client = sinclair-fd
FileSet = "dumpSinclairMail003"
Pool = TapePool
Storage = LTO3
Messages = Standard
Where = /extra/backup/restore
}
Job {
Name = "RestoreSinclairMail004"
Type = Restore
Client = sinclair-fd
FileSet = "dumpSinclairMail004"
Pool = TapePool
Storage = LTO3
Messages = Standard
Where = /extra/backup/restore
}
Schedule {
Name = "Martes"
Run = Differential tue at 3:00
}
Schedule {
Name = "Jueves"
Run = Differential thu at 3:00
}
Schedule {
Name = "Sabado"
Run = Differential sat at 3:00
}
Schedule {
Name = "Miercoles"
Run = Differential wed at 3:00
}
Schedule {
Name = "Viernes"
Run = Differential fri at 3:00
}
FileSet {
Name = "dumpDogo"
Include {
Options {
signature = MD5
compression = GZIP9
}
File = /root
File = /usr/local/majordomo-1.94.5
# File = /var/bacula/
File = /packs/
File = /var/log
File = /extra/backup/crypt
}
}
FileSet {
Name = "dumpAcad"
Include {
Options {
signature = MD5
compression = GZIP9
}
File = /var/log/
File = /var/www/html
File = /var/lib/mysql
File = /extramail/backup/crypt
}
Exclude {
File = /extra/www/html/virtual/claroline*
}
}
FileSet {
Name = "dumpMaxwell"
Include {
Options {
signature = MD5
# compression = GZIP9
}
File = /var/named
File = /var/log
File = /backup/crypt # tiene el /etc, y otros
File = /backup/mysqlbasicdata
File = /usr/sbin/UpFilter
}
}
FileSet {
Name = "dumpTexas"
Include {
Options {
signature = MD5
compression = GZIP9
}
File = /var/log
File = /etc/httpd/conf/httpd.conf
File = /var/named
File = /extra/www
File = /extra/data.FormDB
File = /extra/backup/crypt
}
}
FileSet {
Name = "dumpSinclairMail001"
Include {
Options {
signature = MD5
}
File = /extramail/cyrus
}
Exclude {
File = /extramail/cyrus/[g-z]*
}
}
FileSet {
Name = "dumpSinclairMail002"
Include {
Options {
signature = MD5
}
File = /extramail/cyrus/
}
Exclude {
File = /extramail/cyrus/[a-f]*
File = /extramail/cyrus/[n-z]*
}
}
FileSet{
Name = "dumpSinclairMail003"
Include {
Options {
signature = MD5
}
File = /extramail/cyrus/
}
Exclude {
File = /extramail/cyrus/[a-m]*
File = /extramail/cyrus/[t-z]*
}
}
FileSet {
Name = "dumpSinclairMail004"
Include {
Options {
signature = MD5
# compression = GZIP9
}
File = /extramail/cyrus
File = /extramail/mqueue
File = /usr/lib/cyrus-imapd/
File = /var/lib/imap/
File = /etc/scanup2.cfg
File = /etc/scanup2.regex.cfg
File = /usr/sbin/UpFilter
File = /etc/init.d/UpFilter
File = /etc/init.d/sendmail
File = /etc/mail
}
Exclude {
File = /extramail/cyrus/[a-s]*
}
}
FileSet {
Name = "dumpProbandoLTO3otherclient"
Include {
Options {
signature = MD5
# compression = GZIP9
}
File = /extramail/cyrus
File = /extramail/mqueue
File = /usr/lib/cyrus-imapd/
File = /var/lib/imap/
File = /etc/mail
File = /etc/scanup2.*
File = /usr/sbin/UpFilter
File = /etc/init.d/Upfilter
File = /etc/init.d/sendmail
}
Exclude {
File = /extramail/cyrus/[g-z]*
}
}
Pool {
Name = HDPool
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 2 days # one year
# Accept Any Volume = yes # write on any volume in the pool
}
Pool {
Name = TapePool
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = no # Prune expired volumes
# Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolSabado
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
# Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolSabadoLTO3
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
# Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolMartes
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
# Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolMartesLTO3
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
# Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolJueves
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
#Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolJuevesLTO3
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
#Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = ProbandoLTO3otherclient
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
#Accept Any Volume = no # write on any volume in the pool
}
Pool {
Name = TapePoolAcadM
Pool Type = Backup
Recycle = no # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
# Accept Any Volume = no # write on any volume in the pool
}
Client {
Name = acad-fd
Address = acad
FDPort = 9102
Catalog = MyCatalog
Password----
AutoPrune = yes # Prune expired Jobs/Files
}
Client {
Name = dogo-fd
Address = dogo
FDPort = 9102
Catalog = MyCatalog
Password----
AutoPrune = no # Prune expired Jobs/Files
}
Client {
Name = sinclair-fd
Address = sinclair
FDPort = 9102
Catalog = MyCatalog
Password----
AutoPrune = no # Prune expired Jobs/Files
}
Client {
Name = texas-fd
Address = texas
FDPort = 9102
Catalog = MyCatalog
Password----
AutoPrune = no # Prune expired Jobs/Files
}
Client {
Name = probandolto3-fd
Address = install.wrpowpeowpeopwe.edu
FDPort = 9102
Catalog = MyCatalog
Password----
AutoPrune = no # Prune expired Jobs/Files
}
Storage {
Name = DAT72i
Address = DirectorandStorageServerThatHasDat72 # N.B.
Use a fully qualified name here
SDPort = 9103
Password----
Device = DAT72i # must be same as Device in
Storage daemon
Media Type = DAT72i # must be same as MediaType in
Storage daemon
}
Storage {
Name = LTO3
Address = install.wrpowpeowpeopwe.edu # N.B. Use a
fully qualified name here
SDPort = 9103
Password----
Device = LTO3 # must be same as Device in Storage daemon
Media Type = LTO3 # must be same as MediaType in
Storage daemon
}
Catalog {
Name = MyCatalog
dbname = bacula; DB Address = localhost; user = bacula; password =
"sunguTRULE"
}
Messages {
Name = Daemon
mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) %r\" -s
\"Bacula daemon message\" %r"
mail = r...@wrpowpeowpeopwe.edu = all, !skipped
console = all, !skipped, !saved
append = "/var/bacula/working/log" = all, !skipped
}
Messages {
Name = Standard
mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) %r\" -s
\"Bacula: %t %e of %c %l\" %r"
operatorcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) %r\"
-s \"Bacula: Intervention needed for %j\" %r"
mail = r...@wrpowpeowpeopwe.edu = all, !skipped
operator = r...@wrpowpeowpeopwe.edu = mount
console = all, !skipped, !saved
append = "/var/bacula/working/log" = all, !skipped
}
Console {
Name = DirectorandStorageServerThatHasDat72-mon
Password----
CommandACL = status, .status
}
Storage Server Thas Has LTO3 config files:
bacula-sd.conf
Name = StorageServerThasHasLTO3-sd
WorkingDirectory = "/var/bacula/working"
Pid Directory = "/var/run"
Maximum Concurrent Jobs = 20
}
Director {
Name = DirectorandStorageServerThatHasDat72-dir-dir
Password----
}
Director {
Name = DirectorandStorageServerThatHasDat72-dir-mon
Password----
Monitor = yes
}
Device {
Name = LTO3
Media Type = LTO3
AlwaysOpen = yes;
RemovableMedia = yes;
Maximum Spool Size = 4gb
Maximum Job Spool Size = 3gb
Spool Directory = /var/spool/bacula
Maximum Network Buffer Size = 10240
}
Messages {
Name = Standard
director = StorageServerThasHasLTO3-dir = all
}
mt -f /dev/nst0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x44 (no translation).
Soft error count since last status=0
General status bits on (41010000):
BOT ONLINE IM_REP_EN
Product Type: Disk Drive
Vendor ID: 'ServeRA '
Product ID: 'raid 1 '
Revision: 'V1.0'
Attached Changer: No
SerialNumber: '7E08591F'
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x37
BlockSize: 3748166
DataCompEnabled: no
DataCompCapable: no
DataDeCompEnabled: no
CompType: 0x35393146
DeCompType: 0x300000
Thanks in advance!!
--
--
Open Kairos http://www.openkairos.com
Watch More TV http://sebelk.blogspot.com
Sergio Belkin -
------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
