On 2/12/2019 2:12 PM, mark.berg...@uphs.upenn.edu wrote:
We run the bacula-dir and bacula-sd (v9.4.1) on a CentOS6 server
with multiple network interfaces. I'm seeing a problem with a new client
that I believe is related to asymetric routing and maybe the use of the
server's IP to generate the MD5 CRAM digest.
Maybe to the problem being asymmetric routing (see below), no to the IP
being used to calculate the digest.
Here's a simplified network diagram, best viewed with a fixed-width font:
[client 1] [client 2]
| |
| |
| +-----------------------+
| | |
{192.168.1.0/24} {172.16.1.0/24} |
| | |
| | |
________________________________ |
| 192.168.1.159 172.16.1.159 | |
| | |
| bacula-dir & bacula-sd server | |
| | |
| 10.20.0.159 | |
+-------------------------------+ |
| |
| |
{10.20.0.0/16} |
| |
+---------------+ {172.16.1.0/24}
| | |
_______|______ ___|__________ |
|10.20.0.10 | | default GW | |
| new client | | NAT server |------+
+------------+ +-------------+
Bacula is able to backup from legacy client1 and client2 machines with
no problem.
The client bacula-fd.conf files are identical on each client, except
for the Name field for the FileDaemon itself.
On the bacula server, 10.20.0.159 is a virtual (secondary) IP.
I've added a new client, on a new network (10.20.0.0/16). The new client
is single-homed, with a default route via a server that does NAT for
outbound traffic.
The bacula server is able to communicate with new_client via the 10.20.0.0
network.
The new_client was initially able to communicate with the bacula server
through the default gateway and NAT to the bacula server at 172.16.1.159.
I have created iptables DNAT rules to rewrite outbound traffic from the
new_client that is destined for the bacula server on 172.16.1.159 so
that traffic goes to 10.20.0.159.
Why? If the bacula-dir/bacula-sd host has a 10.20.0.0 address, then why
not just use the 10.20.0.0 network for all Bacula traffic between hosts
on the 10.20.0.0 network? If the problem is that DNS is returning the
172.16.1.159 address for the Bacula server's hostname, then it is a DNS
problem, rather than a networking problem, and no DNAT is required.
Instead, put an entry for the Bacula server's hostname in new_client's
/etc/hosts file, else fiddle with views in the Bind DNS config so that
hosts on the 10.20.0.0 network get the 10.20.0.159 address, hosts on the
172.16.1.0 network get the 172.16.1.159 address, and hosts on the
192.168.1.0 network. get the 192.168.1.159 address.
Now, new_client sends all traffic to the bacula server server via 10.20.0.0.
I haven't looked at the source code, but I'm wondering if the problem is
not in the network traffic itself, but in the authentication challenge. Is
the challenge based on the bacula-dir server's hostname or IP? It appears
as if the bacula server identifies itself as 172.16.1.159 but the client
sees traffic from the server on 10.20.0.159.
Here are the server & client debugging outputs:
------------- Server -----------------
1 [root@bacula-server ~]# /opt/bacula/bin/bconsole
2 Connecting to Director bacula-server:9101
3 bacula-server: bsock.c:851-0 socket=8 who=client host=10.20.0.159
port=63069
4 bacula-server: job.c:1767-0 wstorage=neoxl80
5 bacula-server: job.c:1776-0 wstore=neoxl80 where=Pool resource
6 bacula-server: job.c:1430-0 JobId=0 created
Job=-Console-.2019-02-11_18.42.49_02
7 bacula-server: cram-md5.c:69-0 send: auth cram-md5 challenge
<757376812.1549928569@bacula-server> ssl=0
8 bacula-server: cram-md5.c:133-0 cram-get received: auth cram-md5
<2004139279.1549928569@bconsole> ssl=0
9 bacula-server: cram-md5.c:157-0 sending resp to challenge:
nH+CS8krrl+Qg9+3F/+TZA
10 1000 OK: 103 bacula-server Version: 9.4.1 (20 December 2018)
11 Enter a period to cancel a command.
12 *estimate
13 Select Job resource (1-72): 9
14 bacula-server: ua_cmds.c:2608-0 UA Open database
15 Using Catalog "MyCatalog"
16 bacula-server: job.c:1298-0 Created Client new_client record 56
17 bacula-server: fd_cmds.c:314-0 Level=F last start time= job=
18 Connecting to Client new_client at 10.20.20.1:9102
19 bacula-server: bsockcore.c:299-0 Current 10.20.20.1:9102 All
10.20.20.1:9102
20 bacula-server: bsockcore.c:228-0 who=Client: new_client host=10.20.20.1
port=9102
21 bacula-server: bsockcore.c:411-0 OK connected to server Client:
new_client 10.20.20.1:9102.
22 bacula-server: fd_cmds.c:116-0 Opened connection with File daemon
23 bacula-server: authenticate.c:215-0 Sent: Hello Director bacula-server
calling 103
24 bacula-server: authenticate.c:237-0 cram_get_auth failed for Client:
new_client
25 bacula-server: authenticate.c:241-0 Director and File daemon passwords
or names not the same.
26 bacula-server: fd_cmds.c:123-0 Authentication error with FD.
27 Failed to connect to Client.
28 You have messages.
29 *
--------------------------------------
------------- Client -----------------
1 bacula-fd: address_conf.c:274-0 Initaddr 0.0.0.0:9102
2 new_client: jcr.c:131-0 read_last_jobs seek to 192
3 new_client: jcr.c:138-0 Read num_items=0
4 new_client: plugins.c:97-0 load_plugins
5 new_client: plugins.c:136-0 Found plugin: name=bpipe-fd.so len=11
6 new_client: filed.c:270-0 filed: listening on port 9102
7 new_client: bnet_server.c:86-0 Addresses 0.0.0.0:9102
8 new_client: bsock.c:847-0 socket=4 who=client host=172.16.1.159
port=22958
9 new_client: job.c:317-0 <dird: Hello Director bacula_server calling 103
10 new_client: job.c:340-0 Executing Hello Dir Hello Director
bacula_server calling 103 command.
11 new_client: cram-md5.c:69-0 send: auth cram-md5 challenge
<1469168970.1549998494@new_client> ssl=0
12 new_client: cram-md5.c:71-0 Send challenge comm error. ERR=Connection
reset by peer
13 new_client: authenticate.c:101-0 cram_auth challenge failed for
Director client
14 new_client: Fatal Error at authenticate.c:105 because:
15 Incorrect password given by Director at client.
16 new_client: job.c:343-0 Quit command loop. Canceled=0
17 new_client: job.c:467-0 Calling term_find_files
18 new_client: job.c:470-0 Done with term_find_files
19 new_client: job.c:473-0 Done with free_jcr
------------------------------------------------
In particular, note Line 8, where the bacula server seems to identify
itself as 172.16.1.159 (the primary interface), even though the traffic
was sent via the 10.20.0.0 network.
Any thoughts?
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
