Hi Pann. See comments inline below.
On 5/5/2016 4:29 AM, Pann Tolk wrote:
Hello Guys,
I'm new to Bacula. And it's my first time trying to config Bacula
7.4.0 with vchanger 1.0.1 for my home network.
I have configured Bacula and vchanger following the vchanger 1.0.1
documentation included in the rpm package (vchangerHowto.html)
My hardware setup is as below:
|-SATA_Port_1 -| 2-slots | < Multiple 2TB SATA harddisks, each
HOST -| | hotplug |<== < formated with single ext4 partition
|-SATA_Port_2 -| dock | < (with unique UUID) for the
respective
< filesystem to be used as magazines.
The above 2-slots hotplug dock will allow me insert or remove 2
physical SATA drives (and therefore, 2 magazines) at a time.
For each magazine, I intend to make each volume file to be 100GB.
Here is my host system software setup:
- Fedora 23
- Bacula 7.4.0 (installed from Fedora repository)
Both Bacula Director and Bacula Storage Daemon are installed on the
same host
(above). It will be used to backup other clients on the network.
- vchanger-1.0.1-1.el7.x86_64.rpm (downloaded from Sourceforge:
https://sourceforge.net/projects/vchanger/files/RPM/)
I have the following questions and problem:
1. With regards to using 2 magazines simultaneously, are the
following correct?
/etc/bacula/bacula-sd.conf:
===========================
[root@deneb ~]# cat /etc/bacula/bacula-sd.conf | grep -v "^[#]"
Storage { # definition of myself
Name = bacula-sd
SDPort = 9103 # Director's port
WorkingDirectory = "/var/spool/bacula"
Pid Directory = "/var/run"
Maximum Concurrent Jobs = 20
}
Director {
Name = bacula-dir
Password = "xxxxxxxxxxxxxxxx"
}
Autochanger {
Name = SATA-vchanger01
Device = SATA-vchanger01-drive-0
Changer Command = "vchanger %c %o %S %a %d"
Changer Device = "/etc/vchanger/vchanger01.conf"
}
Device {
Name = SATA-vchanger01-drive-0
Drive Index = 0
Autochanger = yes
Device Type = File
Media Type = File
Removable Media = no
Random Access = yes
Maximum Concurrent Jobs = 1
Archive Device = "/var/spool/vchanger/vchanger01/drive0"
}
Messages {
Name = Standard
director = bacula-dir = all
}
[root@deneb ~]#
/etc/vchanger/vchanger01.conf:
==============================
[root@deneb ~]# cat /etc/vchanger/vchanger01.conf | grep -v "^[#]"
Storage Resource = "vchanger01"
User = bacula
group = tape
work dir = "/var/spool/vchanger/vchanger01"
logfile = "/var/log/vchanger/vchanger01.log"
log level = 3
bconsole = "/usr/sbin/bconsole"
bconsole config = /etc/bacula/bconsole.conf
default pool = "Scratch"
magazine = UUID:1fde15be-83a4-4b67-accc-378d5fa10ea7
magazine = UUID:412060d2-7be4-4719-8089-2ee6cfd28624
magazine = UUID:20d986c1-9391-467e-9f73-232b53fba377
magazine = UUID:fa48ed8d-2e63-4c27-8670-c943822d2cd0
2. I followed the vchanger documentation (vchangerHowto.html) to
setup vchanger.
I have successfully configured udev and autofs. udev is able to mount
the filesystems when I insert the drives. And, autofs is able to
mount it to the mountpoint /mnt/vchanger
You should not configure for both autofs and udev at the same time, but
rather one or the other.
I have also successfully created volumes in the first 2 magazines.
[root@deneb ~]# vchanger /etc/vchanger/vchanger01.conf listmags
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
0:18:1:/mnt/vchanger/1fde15be-83a4-4b67-accc-378d5fa10ea7
1:18:19:/mnt/vchanger/412060d2-7be4-4719-8089-2ee6cfd28624
2:::
3:::
[root@deneb ~]# vchanger /etc/vchanger/vchanger01.conf list
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
device-enumerator: scanning /sys/bus
device-enumerator: scanning /sys/class
1:vchanger01_0_0
2:vchanger01_0_1
3:vchanger01_0_10
4:vchanger01_0_11
5:vchanger01_0_12
6:vchanger01_0_13
7:vchanger01_0_14
8:vchanger01_0_15
9:vchanger01_0_16
10:vchanger01_0_17
11:vchanger01_0_2
12:vchanger01_0_3
13:vchanger01_0_4
14:vchanger01_0_5
15:vchanger01_0_6
16:vchanger01_0_7
17:vchanger01_0_8
18:vchanger01_0_9
19:vchanger01_1_0
20:vchanger01_1_1
21:vchanger01_1_10
22:vchanger01_1_11
23:vchanger01_1_12
24:vchanger01_1_13
25:vchanger01_1_14
26:vchanger01_1_15
27:vchanger01_1_16
28:vchanger01_1_17
29:vchanger01_1_2
30:vchanger01_1_3
31:vchanger01_1_4
32:vchanger01_1_5
33:vchanger01_1_6
34:vchanger01_1_7
35:vchanger01_1_8
36:vchanger01_1_9
37:
[root@deneb ~]# echo "status slots storage=vchanger01 drive=0" | bconsole
Connecting to Director deneb.pt.local:9101
1000 OK: 102 bacula-dir Version: 7.4.0 (16 January 2016)
Enter a period to cancel a command.
status slots storage=vchanger01 drive=0
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
Connecting to Storage daemon vchanger01 at deneb.pt.local:9103 ...
3306 Issuing autochanger "slots" command.
Device "SATA-vchanger01" has 0 slots.
No slots in changer to scan.
Questions:
==========
2a. Why did vchanger list and listmags spits out "device-enumerator"
lines?
Those are generated by systemd. My best guess is that they are triggered
by vchanger's use of the libudev function udev_enumerate_scan_devices()
to scan for mounted filesystems by UUID. But systemd is a total enigma
to me, so I am not sure why this would be the case.
2b. bconsole throws out the lines below when I issue "status slots".
Did I misconfigured something?
Device "SATA-vchanger01" has 0 slots.
No slots in changer to scan.
This is very likely one of two things (or both); a race condition bug in
vchanger or vchanger is incompatible with the installed version of
libudev..
The vchanger bug is due to vchanger automatically issuing 'update slots'
commands via bconsole when it detects a change in the volume files it
has access to. Vchanger is invoking bconsole in a child process. The
child bconsole process executes a 'update slots' command that in turn
causes bacula-sd to invoke a few other instances of vchanger with the
SLOTS and LIST commands, which is how bacula-sd queries autochangers to
obtain the number of slots and current volume barcodes in those slots.
Vchanger uses a lockfile to serialize the simultaneous instances, but
some bug in vchanger is causing a race for the lockfile (I think).
The simple workaround to determine if the vchanger bug is causing your
problem, is to change the bconsole="/usr/sbin/bconsole" line in
vchanger.conf to bconsole="". This will prevent vchanger from issuing
bconsole commands at all. Of course you will then have to issue the
update slots command in bconsole manually whenever anything changes with
the magazine drives. You may need to manually delete any leftover stale
.lock files in the vchanger work directory.
Another possibility is that there has been some change in recent
releases of libudev that causes vchanger to not find filesystem
mountpoints by UUID. That would perhaps explain the messages from
systemd. The workaround for that is to make sure the drives are mounted,
either by udev or manually, and then specify the magazines as a
mountpoint path in vchanger.conf, rather than as a UUID. When specified
as a full path, vchanger just assumes the volumes are in that directory
and does not call any libudev functions.
2c. Also, when I issue "status storage" in bconsole, I get following:
*status storage
...
...
Device status:
Autochanger "SATA-vchanger01" with devices:
"SATA-vchanger01-drive-0" (/var/spool/vchanger/vchanger01/drive0)
Device file: "SATA-vchanger01-drive-0"
(/var/spool/vchanger/vchanger01/drive0) is not open.
Drive 0 is not loaded.
I have searched Google and still not able to find out why / what's
wrong with my setup. Appreciate if someone can help me out.
Thanks in advance.
PanTor
=====================================================================
Below are my settings and config files
=====================================================================
--------------------------------
SELINUX Setting
--------------------------------
[root@deneb ~]# /usr/sbin/sestatus
SELinux status: disabled
[root@deneb ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected
processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
-----------------------------------
Bacula Director and SD Running As
-----------------------------------
[root@deneb ~]# ps -aux | grep bacula
bacula 12628 0.0 0.5 365132 11124 ? Ssl 14:10 0:00
/usr/sbin/bacula-dir -f -c /etc/bacula/bacula-dir.conf -u bacula -g bacula
bacula 12640 0.0 0.3 327868 7764 ? Ssl 14:10 0:00
/usr/sbin/bacula-sd -f -c /etc/bacula/bacula-sd.conf -u bacula -g tape
-----------------------------------
Config for Bacula
-----------------------------------
[root@deneb ~]# ls -l /etc/bacula/
total 36
-rw-r--r-- 1 bacula bacula 9419 May 3 13:56 bacula-dir.conf
-rw-r--r-- 1 bacula bacula 8944 May 5 09:58 bacula-sd.conf
-rw-r--r-- 1 bacula bacula 254 Apr 26 11:54 bconsole.conf
-rw-r----- 1 bacula bacula 312 Jan 26 04:26 query.sql
--------------------------------
/etc/bacula/bacula-dir.conf
--------------------------------
[root@deneb ~]# cat /etc/bacula/bacula-dir.conf | grep -v "^[#]"
Director { # define myself
Name = bacula-dir
DIRport = 9101 # where we listen for UA connections
QueryFile = "/etc/bacula/query.sql"
WorkingDirectory = "/var/spool/bacula"
PidDirectory = "/var/run"
Maximum Concurrent Jobs = 20
Password = "xxxxxxxxxxxxxxxx" # Console password
Messages = Daemon
}
JobDefs {
Name = "DefaultJob"
Type = Backup
Level = Incremental
Client = bacula-fd
FileSet = "Full Set"
Schedule = "WeeklyCycle"
Storage = vchanger01
Messages = Standard
Pool = File
SpoolAttributes = yes
Priority = 10
Write Bootstrap = "/var/spool/bacula/%c.bsr"
}
Job {
Name = "BackupClient1"
JobDefs = "DefaultJob"
}
Job {
Name = "BackupCatalog"
JobDefs = "DefaultJob"
Level = Full
FileSet="Catalog"
Schedule = "WeeklyCycleAfterBackup"
# This creates an ASCII copy of the catalog
# Arguments to make_catalog_backup.pl
<http://make_catalog_backup.pl> are:
# make_catalog_backup.pl <http://make_catalog_backup.pl> <catalog-name>
RunBeforeJob = "/usr/libexec/bacula/make_catalog_backup.pl
<http://make_catalog_backup.pl> MyCatalog"
# This deletes the copy of the catalog
RunAfterJob = "/usr/libexec/bacula/delete_catalog_backup"
Write Bootstrap = "/var/spool/bacula/%n.bsr"
Priority = 11 # run after main backup
}
Job {
Name = "RestoreFiles"
Type = Restore
Client=bacula-fd
FileSet="Full Set"
Storage = vchanger01
Pool = File
Messages = Standard
Where = /tmp/bacula-restores
}
FileSet {
Name = "Full Set"
Include {
Options {
signature = MD5
}
File = /usr/sbin
}
Exclude {
File = /var/spool/bacula
File = /tmp
File = /proc
File = /tmp
File = /sys
File = /.journal
File = /.fsck
}
}
Schedule {
Name = "WeeklyCycle"
Run = Full 1st sun at 23:05
Run = Differential 2nd-5th sun at 23:05
Run = Incremental mon-sat at 23:05
}
Schedule {
Name = "WeeklyCycleAfterBackup"
Run = Full sun-sat at 23:10
}
FileSet {
Name = "Catalog"
Include {
Options {
signature = MD5
}
File = "/var/spool/bacula/bacula.sql"
}
}
Client {
Name = bacula-fd
Address = episilon.pt.local
FDPort = 9102
Catalog = MyCatalog
Password = "xxxxxxxxxxxxxxxx" # password for FileDaemon
File Retention = 60 days # 60 days
Job Retention = 6 months # six months
AutoPrune = yes # Prune expired Jobs/Files
}
Storage {
Name = vchanger01
Address = deneb.pt.local # N.B. Use a fully qualified name here
SDPort = 9103
Password = "xxxxxxxxxxxxxxxx"
Device = SATA-vchanger01
Media Type = File
Autochanger = yes
Maximum Concurrent Jobs = 10 # run up to 10 jobs a the same time
}
Catalog {
Name = MyCatalog
dbname = "bacula"; dbuser = "bacula"; dbpassword = "xxxxxxxxxxxxxxxx"
}
Messages {
Name = Standard
mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\"
-s \"Bacula: %t %e of %c %l\" %r"
operatorcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\)
\<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"
mail = root@localhost = all, !skipped
operator = root@localhost = mount
console = all, !skipped, !saved
append = "/var/log/bacula/bacula.log" = all, !skipped
catalog = all
}
Messages {
Name = Daemon
mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\"
-s \"Bacula daemon message\" %r"
mail = root@localhost = all, !skipped
console = all, !skipped, !saved
append = "/var/log/bacula/bacula.log" = all, !skipped
}
Pool {
Name = Default
Pool Type = Backup
Recycle = yes # Bacula can automatically
recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # one year
Maximum Volume Bytes = 100G # Limit Volume size to something
reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
}
Pool {
Name = File
Pool Type = Backup
Recycle = yes # Bacula can automatically
recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # one year
Maximum Volume Bytes = 100G # Limit Volume size to something
reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
Label Format = "Vol-" # Auto label
}
Pool {
Name = Scratch
Pool Type = Backup
}
--------------------------------
/etc/bacula/bacula-sd.conf
--------------------------------
[root@deneb ~]# cat /etc/bacula/bacula-sd.conf | grep -v "^[#]"
Storage { # definition of myself
Name = bacula-sd
SDPort = 9103 # Director's port
WorkingDirectory = "/var/spool/bacula"
Pid Directory = "/var/run"
Maximum Concurrent Jobs = 20
}
Director {
Name = bacula-dir
Password = "xxxxxxxxxxxxxxxx"
}
Autochanger {
Name = SATA-vchanger01
Device = SATA-vchanger01-drive-0
Changer Command = "vchanger %c %o %S %a %d"
Changer Device = "/etc/vchanger/vchanger01.conf"
}
Device {
Name = SATA-vchanger01-drive-0
Drive Index = 0
Autochanger = yes
Device Type = File
Media Type = File
Removable Media = no
Random Access = yes
Maximum Concurrent Jobs = 1
Archive Device = "/var/spool/vchanger/vchanger01/drive0"
}
Messages {
Name = Standard
director = bacula-dir = all
}
--------------------------------
/etc/bacula/bconsole.conf
--------------------------------
[root@deneb ~]# cat /etc/bacula/bconsole.conf | grep -v "^[#]"
Director {
Name = bacula-dir
DIRport = 9101
address = deneb.pt.local
Password = "xxxxxxxxxxxxxxxx"
--------------------------------
/etc/vchanger/vchanger01.conf
--------------------------------
[root@deneb ~]# cat /etc/vchanger/vchanger01.conf | grep -v "^[#]"
Storage Resource = "vchanger01"
User = bacula
group = tape
work dir = "/var/spool/vchanger/vchanger01"
logfile = "/var/log/vchanger/vchanger01.log"
log level = 3
bconsole = "/usr/sbin/bconsole"
bconsole config = /etc/bacula/bconsole.conf
default pool = "Scratch"
magazine = UUID:1fde15be-83a4-4b67-accc-378d5fa10ea7
magazine = UUID:412060d2-7be4-4719-8089-2ee6cfd28624
magazine = UUID:20d986c1-9391-467e-9f73-232b53fba377
magazine = UUID:fa48ed8d-2e63-4c27-8670-c943822d2cd0
--------------------------------
Files Ownership and Permissions
--------------------------------
[root@deneb ~]# ls -l /mnt/vchanger/
total 8
drwxr-x--- 3 bacula tape 4096 May 4 19:34
1fde15be-83a4-4b67-accc-378d5fa10ea7
drwxr-x--- 3 bacula tape 4096 May 4 19:34
412060d2-7be4-4719-8089-2ee6cfd28624
[root@deneb ~]# ls -l /mnt/vchanger/1fde15be-83a4-4b67-accc-378d5fa10ea7/
total 16
drwxr-x--- 2 bacula tape 16384 Apr 27 13:57 lost+found
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_0
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_1
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_10
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_11
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_12
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_13
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_14
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_15
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_16
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_17
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_2
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_3
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_4
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_5
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_6
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_7
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_8
-rw-r--r-- 1 bacula tape 0 May 4 19:34 vchanger01_0_9
[root@deneb ~]# ls -l /etc/vchanger
total 4
-rw-r--r-- 1 root root 3754 May 5 10:12 vchanger01.conf
[root@deneb ~]# ls -l /etc/vchanger/vchanger01.conf
-rw-r--r-- 1 root root 3754 May 5 10:12 /etc/vchanger/vchanger01.conf
[root@deneb ~]# ls -l /var/spool/bacula
total 8
-rw-r----- 1 bacula bacula 196 May 5 14:10 bacula-dir.9101.state
-rw------- 1 bacula bacula 0 May 4 14:22 bacula-dir.conmsg
-rw-r----- 1 bacula tape 196 May 5 14:10 bacula-sd.9103.state
[root@deneb ~]# ls -l /var/spool/vchanger
total 4
drwxr-x--- 2 bacula tape 4096 May 5 15:17 vchanger01
[root@deneb ~]# ls -l /var/spool/vchanger/vchanger01
total 12
-rw-r----- 1 bacula tape 47 May 5 15:17 bay_state-0
-rw-r----- 1 bacula tape 48 May 5 15:17 bay_state-1
-rw-r----- 1 bacula tape 17 May 5 15:17 dynamic.conf
------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
