Hello,
I am trying to setup Amanda 3.3.9 on two Linux (ArchLinux) systems. I have
succeeded to get a basic backup-recovery cycle running on one machine, but
failed to reproduce the setup on the other machine. After staring on my screen
for two days, I have to admit that I cannot see which mistake I have made so
that Amanda fails on one machine but works on the other one.
But first the details:
On my desktop machine running a current ArchLinux where I succeeded to run
Amanda, I have compile Amanda 3.3.9 from source using the following invocations:
./autogen
./configure --without-force-uid
--with-security-file="${DESTDIR}/etc/amanda/amanda-security.conf"
--with-single-userid --prefix=${DESTDIR}${INSTALLDIRECTORY} --disable-rpath
--disable-nls --disable-s3-device --disable-dvdrw-device --disable-ndmp-device
--with-user=${AMANDA_USER} --with-group=${AMANDA_GROUP}
--with-index-server=localhost --with-readline --without-ipv6
--sysconfdir=${DESTDIR}/etc --with-configdir=${DESTDIR}/etc/amanda
--localstatedir=${DESTDIR}/var
--with-amperldir=${DESTDIR}${INSTALLDIRECTORY}/share/perl5/site_perl
make && make install
As this is supposed to be a test installation, I set various variables used in
above invocations as follows:
AMANDA_USER=$(id -u -n)
AMANDA_GROUP=$(id -g -n)
DESTDIR="/tmp/amanda-destdir"
INSTALLDIRECTORY="/usr"
To be able to run Amanda from command line, I set variables PATH, PERL5LIB, and
LD_LIBRARY_PATH to cover the locations where Amanda got installed to.
You may have also noticed that I am not using a special user (just my regular
account) as well as using the configure flag '--with-single-userid'.
I created the following directories:
/tmp/backup/vtapes/slot{1,2,3,4,5}
/tmp/backup/holding
/tmp/backup/state/{curinfo,log,index}
/tmp/amanda-destdir/var/amanda/gnutar-lists
.. and used the following configuration files, inspired by the GSWA Wiki
documentation and examples ('f15h' is my regular username):
---8<--- /tmp/amanda-destdir/etc/amanda/mybackup/amanda.conf ---8<---
org "LIM" # your organization name for reports
displayunit "k" # k=kilo, m=mega, g=giga, t=tera
infofile "/tmp/backup/state/curinfo"
logdir "/tmp/backup/state/log"
indexdir "/tmp/backup/state/index"
dumpuser "f15h"
runtapes 1 # number of tapes to be used in a single run of amdump
# tpchanger ('tape changer' can alternate between multiple tapes (like a robot)
# specifically 'chg-disk' is able to switch between multiple directories
representing virtual tapes
tpchanger "chg-disk:/tmp/backup/vtapes"
labelstr "^User_f15h_Backup_[0-9][0-9]$" # regular expression, to recognize
labels
# No auto-labeling, should have been done using 'amlabel' before running
'amdump'
### autolabel "User_f15h_Backup_%%" EMPTY VOLUME_ERROR
# The maximum time between full backups (without specifying week day):
dumpcycle 3 days
# How often is 'amdump' run within the time period of 'dumpcycle'?
# For example, if dumpcycle is '4 weeks' and amdump is run on every weekday
(mo-fr),
# 'runspercycle' should be 20 (4*5)
runspercycle 1
# Number of tapes in use, here the number of vtapes/slot* directories
# Must be at least 'runspercycle' (assuming one tape is sufficient for a full
backup)
tapecycle 3
amrecover_changer "changer" # 'amrecover' will use the changer if you restore
from this device
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
# By default, Amanda can only track at most one run per calendar day. When
# the usetimestamps option is enabled, however, Amanda can track as many
# runs as you care to make.
usetimestamps yes
tapetype "TEST-TAPE"
define tapetype TEST-TAPE {
length 1000 mbytes
filemark 4 kbytes
}
define dumptype simple-gnutar-local {
auth "local"
index yes
record no
compress none
program "GNUTAR"
}
# Holding disk is buffer between original data and 'tape', thus a staging area
for backups
holdingdisk hd1 {
comment "Holding disk"
directory "/tmp/backup/holding"
use -500 Mb # how much space can we use on it; a non-positive value means:
use all space but that value
chunksize 1 mbyte
}
---8<---8<---8<---
---8<--- /tmp/amanda-destdir/etc/amanda/mybackup/disklist ---8<---
# HOST DIRECTORY-ON-HOST-TO-BACKUP DUMPTYPE
localhost /tmp/example-data simple-gnutar-local
---8<---8<---8<---
Directory /tmp/example-data is filled with some files to see if backup and
restore is working.
---8<--- /tmp/amanda-destdir/etc/amanda/amanda-client.conf ---8<---
index_server "localhost"
tapedev "changer"
auth "local" # no special authentication, rely on operating system
---8<---8<---8<---
Labels are set on the virtual tapes as follow:
cd /tmp/backup/vtapes && for slotdir in slot[1-9]* ; do export
slotnumber=${slotdir/slot} ; amlabel mybackup User_f15h_Backup_$(printf "%02d"
${slotnumber}) slot ${slotnumber} ; done
The first test is to run 'amcheck':
---8<---8<---8<---
$ amcheck mybackup
Amanda Tape Server Host Check
-----------------------------
Holding disk /tmp/backup/holding: 7258112 KB disk space available, using
6746112 KB
slot 1: volume 'User_f15h_Backup_01'
Will write to volume 'User_f15h_Backup_01' in slot 1.
NOTE: skipping tape-writable test
NOTE: host info dir /tmp/backup/state/curinfo/localhost does not exist
NOTE: it will be created on the next run.
NOTE: index dir /tmp/backup/state/index/localhost does not exist
NOTE: it will be created on the next run.
Server check took 0.116 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 1.030 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
---8<---8<---8<---
.. and 'amreport':
---8<---8<---8<---
$ amreport mybackup
nothing to report on!
---8<---8<---8<---
(well, nothing had been backup'ed)
Then, I try to trigger a backup, running 'amdump'. No output, so I run
'amcheck' and 'amreport' again:
---8<---8<---8<---
$ amdump mybackup
$ amcheck mybackup
Amanda Tape Server Host Check
-----------------------------
Holding disk /tmp/backup/holding: 6639616 KB disk space available, using
6127616 KB
slot 1: volume 'User_f15h_Backup_01' is still active and cannot be overwritten
slot 2: volume 'User_f15h_Backup_02'
Will write to volume 'User_f15h_Backup_02' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.114 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 1.021 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
$ amreport mybackup
Hostname: desktop
Org : LIM
Config : mybackup
Date : december 11, 2016
These dumps were to tape User_f15h_Backup_01.
The next tape Amanda expects to use is: 1 new tape.
The next new tape already labelled is: User_f15h_Backup_02.
STATISTICS:
Total Full Incr. Level:#
-------- -------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 603.3 603.3 0.0
Original Size (meg) 603.3 603.3 0.0
Avg Compressed Size (%) 100.0 100.0 --
DLEs Dumped 1 1 0
Avg Dump Rate (k/s) 747857 747857 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 603.3 603.3 0.0
Tape Used (%) 60.3 60.3 0.0
DLEs Taped 1 1 0
Parts Taped 1 1 0
Avg Tp Write Rate (k/s) 6177300 6177300 --
USAGE BY TAPE:
Label Time Size % DLEs Parts
User_f15h_Backup_01 0:00 617730K 60.3 1 1
NOTES:
planner: Adding new disk localhost:/tmp/example-data.
taper: Slot 1 with label User_f15h_Backup_01 is usable
taper: tape User_f15h_Backup_01 kb 617730 fm 1 [OK]
DUMP SUMMARY:
DUMPER STATS
TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS
KB/s
--------------------------------8<----------------------8<----------------8<----------------
localhost /tmp/example-data 0 617730 617730 -- 0:01 747807.3 0:00
6177300.0
(brought to you by Amanda version 3.3.9)
---8<---8<---8<---
As seen in the STATISTICS section in above output, about 600 MB have been
backup'ed, which is about right. Indeed, I can recover those files with
'amrecover', although I omit the output here.
Now, my problem starts when I try to configure Amanda in a virtual machine in a
setup closer to the intended deployment target, but still a testing
environment. The guest in the virtual machine is running a current ArchLinux as
well. This ArchLinux instance is my custom-built USB/ISO live system, built and
configured as described here: https://wiki.archlinux.org/index.php/Archiso
Being a read-only live system, I have attached a virtual harddrive to this
virtual machine; this harddrive is mounted under /backup.
The data to be backup'ed is read from /windows/documents.
Again, I start by building Amanda from source on the desktop system, where the
live system's ISO file will be assembled. The instructions look as follows
(should be identical to above, except for different paths and users):
./autogen
./configure --without-force-uid
--with-security-file="/etc/amanda/amanda-security.conf" --with-single-userid
--prefix=${INSTALLDIRECTORY} --disable-rpath --disable-nls --disable-s3-device
--disable-dvdrw-device --disable-ndmp-device --with-user=${AMANDA_USER}
--with-group=${AMANDA_GROUP} --with-index-server=localhost --with-readline
--without-ipv6 --sysconfdir=/etc --with-configdir=/etc/amanda
--localstatedir=/var --with-amperldir=${INSTALLDIRECTORY}/share/perl5/site_perl
.. using the following variables:
AMANDA_USER=amanda (regular user account, UID >= 1000, auto-loggedin on the
live system)
AMANDA_GROUP=amanda (corresponding group)
DESTDIR="/path/to/airootfs" ('airootfs' is the name of the directory during
live system customization that will later become '/' on the running live system)
INSTALLDIRECTORY="/usr" (not '/usr/local')
The configuration files look as follows ('usb' is the name of the
configuration, paths are as seen on a running live system):
---8<--- /etc/amanda/usb/amanda.conf ---8<---
org "LIM" # your organization name for reports
displayunit "k" # k=kilo, m=mega, g=giga, t=tera
infofile "/backup/state/curinfo"
logdir "/backup/state/log"
indexdir "/backup/state/index"
dumpuser "amanda"
runtapes 1 # number of tapes to be used in a single run of amdump
# tpchanger ('tape changer' can alternate between multiple tapes (like a robot)
# specifically 'chg-disk' is able to switch between multiple directories
representing virtual tapes
tpchanger "chg-disk:/backup/vtapes"
labelstr "^User_amanda_Backup_[0-9][0-9]$" # regular expression, to recognize
labels
# No auto-labeling, should have been done using 'amlabel' before running
'amdump'
### autolabel "User_amanda_Backup_%%" EMPTY VOLUME_ERROR
# The maximum time between full backups (without specifying week day):
dumpcycle 3 days
# How often is 'amdump' run within the time period of 'dumpcycle'?
# For example, if dumpcycle is '4 weeks' and amdump is run on every weekday
(mo-fr),
# 'runspercycle' should be 20 (4*5)
runspercycle 1
# Number of tapes in use, here the number of vtapes/slot* directories
# Must be at least 'runspercycle' (assuming one tape is sufficient for a full
backup)
tapecycle 3
amrecover_changer "changer" # 'amrecover' will use the changer if you restore
from this device
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
# By default, Amanda can only track at most one run per calendar day. When
# the usetimestamps option is enabled, however, Amanda can track as many
# runs as you care to make.
usetimestamps yes
tapetype "TEST-TAPE"
define tapetype TEST-TAPE {
length 1000 mbytes
filemark 4 kbytes
}
define dumptype simple-gnutar-local {
auth "local"
index yes
record no
compress none
program "GNUTAR"
}
# Holding disk is buffer between original data and 'tape', thus a staging area
for backups
holdingdisk hd1 {
comment "Holding disk"
directory "/backup/holding"
use -500 Mb # how much space can we use on it; a non-positive value means:
use all space but that value
chunksize 1 mbyte
}
---8<---8<---8<---
---8<--- /etc/amanda/usb/disklist ---8<---
# HOST DIRECTORY-ON-HOST-TO-BACKUP DUMPTYPE
localhost /windows/documents simple-gnutar-local
---8<---8<---8<---
---8<--- /etc/amanda/amanda-client.conf ---8<---
index_server "localhost"
tapedev "changer"
auth "local" # no special authentication, rely on operating system
---8<---8<---8<---
During boot time, a special script is run as root to initialize all directories
for Amanda:
---8<---8<---8<---
AMANDA_USER=amanda
WRITEABLEETCAMANDA=/backup/etc-amanda
READONLYETCAMANDA=/etc/amanda
WRITEABLEVARAMANDA=/backup/var-amanda
READONLYVARAMANDA=/var/amanda
# Create, mount, and populate a writable /etc/amanda directory
# Assumption: /backup/etc-amanda is a writable directory and
# /etc/amanda is read-only but contains valid data
mkdir -p ${WRITEABLEETCAMANDA} ${READONLYETCAMANDA} # in case it does not yet
exist
umount ${READONLYETCAMANDA} 2>/dev/null # in case it is mounted
# If there is no valid content in /backup/etc-amanda/, copy valid, read-only
data from /etc/amanda there
test ! -f ${WRITEABLEETCAMANDA}/amanda-client.conf && test -d
${READONLYETCAMANDA} && cp -r ${READONLYETCAMANDA}/* ${WRITEABLEETCAMANDA}/
# Always use latest, read-only copy of certain configuration files
cp ${READONLYETCAMANDA}/usb/disklist ${READONLYETCAMANDA}/usb/amanda.conf
${WRITEABLEETCAMANDA}/usb/
cp ${READONLYETCAMANDA}/amanda-client.conf ${WRITEABLEETCAMANDA}/
chown -R amanda:amanda ${WRITEABLEETCAMANDA} # fix ownership
chown root:root /etc/amanda/amanda-security.conf
mount -o bind ${WRITEABLEETCAMANDA} ${READONLYETCAMANDA} # mount writable
location /backup/etc-amanda to /etc/amanda, shadowing read-only content
# Create, mount, and populate a writable /var/amanda directory
mkdir -p ${WRITEABLEVARAMANDA} ${READONLYVARAMANDA} # in case it does not yet
exist
mkdir -p ${WRITEABLEVARAMANDA}/gnutar-lists # Create directory that may be
missing
umount ${READONLYVARAMANDA} 2>/dev/null # in case it is mounted
chown -R amanda:amanda ${WRITEABLEVARAMANDA} # fix ownership
mount -o bind ${WRITEABLEVARAMANDA} ${READONLYVARAMANDA} # mount writable
location /backup/var-amanda to /var/amanda, shadowing read-only content
# Create directories that may be missing
mkdir -p /backup/vtapes/slot{1,2,3,4,5} /backup/holding
/backup/state/{curinfo,log,index}
chown -R amanda:amanda /backup/vtapes /backup/holding /backup/state
( cd /backup/vtapes && for slotdir in slot[1-9]* ; do export
slotnumber=${slotdir/slot} ; su -c "amlabel usb User_amanda_Backup_$(printf
"%02d" ${slotnumber}) slot ${slotnumber}" amanda ; done )
su -c "amtape usb reset" amanda
---8<---8<---8<---
Some explanation: To allow for some persistence across reboots, /var/amanda and
/etc/amanda will be bind-mounted from /backup/var-amanda and
/backup/etc-amanda, respectively.
Before mounting /backup/etc-amanda into /etc/amanda, some/all configuration
files are copied from /etc/amanda to /backup/etc-amanda (to be precise, files
'disklist' and 'amanda.conf' as shown above.
Also, under /backup, the directories vtapes/slot{1,2,3,4,5}, holding, and
state/{curinfo,log,index} are created.
All ownerships are set to amanda:amanda.
'amlabel' is used to set the labels for the virtual tapes. Eventually, 'amtape'
is run.
The output of above script, as extract from systemd's logs looks as follows:
---8<---8<---8<---
Dec 11 15:17:37 amandausb systemd[1]: Starting Initialize directories and
configuration for Amanda...
Dec 11 15:17:38 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:38 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:38 amandausb init-amanda[314]: Writing label
'User_amanda_Backup_01'...
Dec 11 15:17:38 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:38 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label
'User_amanda_Backup_02'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label
'User_amanda_Backup_03'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label
'User_amanda_Backup_04'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label
'User_amanda_Backup_05'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: changer is reset
Dec 11 15:17:39 amandausb systemd[1]: Started Initialize directories and
configuration for Amanda.
---8<---8<---8<---
Now I am trying to run 'amcheck usb' and 'amreport usb' as regular user
'amanda' inside the running live system on a fresh /backup (i.e. no data was
ever dumped):
---8<---8<---8<---
$ amreport usb
nothing to report on!
$ amcheck usb
Amanda Tape Server Host Check
-----------------------------
Holding disk /backup/holding: 1007616 KB disk space available, using 495616 KB
slot 1: volume 'User_amanda_Backup_01'
Will write to volume 'User_amanda_Backup_01' in slot 1.
NOTE: skipping tape-writable test
NOTE: host info dir /backup/state/curinfo/localhost does not exist
NOTE: it will be created on the next run.
NOTE: index dir /backup/state/index/localhost does not exist
NOTE: it will be created on the next run.
Server check took 0.141 seconds
Amanda Backup Client Hosts Check
--------------------------------
ERROR: localhost: [/etc/amanda/amanda-security.conf
(/etc/amanda/amanda-security.conf) is not owned by root]
Client check: 1 host checked in 1.026 seconds. 1 problem found.
(brought to you by Amanda 3.3.9)
---8<---8<---8<---
Running 'amdump usb' now, followed by 'amcheck usb' and 'amreport usb', all as
regular user 'amanda':
---8<---8<---8<---
$ amdump usb
$ amreport usb
Hostname: amandausb
Org : LIM
Config : usb
Date : December 11, 2016
These dumps were to tape User_amanda_Backup_01.
The next tape Amanda expects to use is: 1 new tape.
The next new tape already labelled is: User_amanda_Backup_02.
STATISTICS:
Total Full Incr. Level:#
-------- -------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 0.0 0.0 0.0
Original Size (meg) 0.0 0.0 0.0
Avg Compressed Size (%) 100.0 100.0 --
DLEs Dumped 1 1 0
Avg Dump Rate (k/s) 714.3 714.3 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) Amanda Tape Server Host Check
-----------------------------
Holding disk /backup/holding: 1007616 KB disk space available, using 495616 KB
slot 1: volume 'User_amanda_Backup_01' is still active and cannot be overwritten
slot 2: volume 'User_amanda_Backup_02'
Will write to volume 'User_amanda_Backup_02' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.158 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 2.029 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
0.0 0.0 0.0
DLEs Taped 1 1 0
Parts Taped 1 1 0
Avg Tp Write Rate (k/s) 100.0 100.0 --
USAGE BY TAPE:
Label Time Size % DLEs Parts
User_amanda_Backup_01 0:00 10K 0.0 1 1
NOTES:
planner: Adding new disk localhost:/windows/documents.
taper: Slot 1 with label User_amanda_Backup_01 is usable
taper: tape User_amanda_Backup_01 kb 10 fm 1 [OK]
DUMP SUMMARY:
DUMPER STATS TAPER
STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS
KB/s
--------------------------------- ---------------------- --------------
-------------
localhost /windows/documents 0 10 10 -- 0:00 682.7 0:00
100.0
(brought to you by Amanda version 3.3.9)
$ amcheck usb
Amanda Tape Server Host Check
-----------------------------
Holding disk /backup/holding: 1007616 KB disk space available, using 495616 KB
slot 1: volume 'User_amanda_Backup_01' is still active and cannot be overwritten
slot 2: volume 'User_amanda_Backup_02'
Will write to volume 'User_amanda_Backup_02' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.158 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 2.029 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
---8<---8<---8<---
As can be seen in the output, 'Output Size (meg)' and 'Original Size (meg)',
despite having data in the directory to be backup'ed:
---8<---8<---8<---
$ ls -l /windows/documents
total 8256
-rw-r--r-- 1 root root 30 Dec 11 18:34 date.txt
-rw-r--r-- 1 root root 53103 Dec 11 18:34 dmesg.txt
-rw-r--r-- 1 root root 11487 Dec 11 18:34 ps-ax.txt
-rw-r--r-- 1 root root 8388608 Dec 11 18:34 rand.bin
---8<---8<---8<---
So, the question is: what am I doing wrong in my second, live CD/USB setup? I
cannot see any obvious problems, so I am open for suggestions!
Looking forward to your help!
Bye,
Thomas
