Hello list,
using the excellent hoststated(8), I have build a reverse proxy
"yellow" with the snapshot from around November 3rd on an i386 box
which is connected to "white" (another OpenBSD host) on the left via
fxp2, and to "orange" (a Linux host) to its right via fxp3.
(white.fxp3) and (fxp2.yellow) are in subnet 10.1.1.0/24, whereas
(yellow.fxp3) and (eth0.orange) are in 10.2.2.0(24.
...---tun0.white.fxp3---fxp2.yellow.fxp3---eth0.orange
hoststated on "yellow" has several relays that listen on various ports
on its right (fxp3) and relays them via (fxp2) to left to "white",
such as for ex. (fxp3):6000/tcp (for X11 relaying using the generic
TCP relay), or (fxp3):53/udp (for DNS relaying using the DNS/UDP
relay). When "orange" issues X11 connections from its (eth0.orange) to
(yellow.fxp3), "yellow" relays those connections perfectly to "white"
using (fxp2.yellow) as source and (white.fxp3) as destination
addresses. This also works fine for all other TCP relays that I tried.
However, the UDP/DNS relay behaves slightly different. It relays DNS
request to the left with (yellow.fxp3) (note: fxpTHREE) as source
address, unlike TCP relays and not as I had expected, with
(fxp2.yellow) (note: fxpTWO) as source address.
Yet, I am not sure if this is intended behaviour, or a bug that
surfaces with my particular use of hoststated to insulating two
subnetworks.
This unexpected behaviuor of the DNS/UDP relay then causes routing
problems as the "white" server is by default unaware of a route for
the source address (yellow.fxp3). So DNS responses from "white" do not
get routed back to "yellow". Of course, I could add a route on "white"
as a workaround. But it would be more elegant if the DNS/UDP relay
would behave the same way as the TCP relays do. In order to figure out
if I misunderstand this, I have started to read the source of
relay_dns_request() in /src/usr.sbin/hoststated/relay_udp.c of
-current. I am grateful for any hints with the cluestick. Also, I am
happy to provide more info, or to apply and test patches.
Thanks,
Rolf
P.S. For those who wonder what this is all about, "orange" is an
application server in a secure zone, and we need to get data across
two security boundaries safely to "white", which is in fact a VPN
tunnel endpoint in a less secure zone. The "---" in the figure above
are actually large networks that are separated by firewalls. However,
in my lab setup these are simple cross-over Ethernet cables :-)
[EMAIL PROTECTED]:etc]# dmesg
OpenBSD 4.2-current (GENERIC) #476: Fri Nov 2 14:41:26 MDT 2007
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Mobile Intel(R) Celeron(TM) CPU 650MHz ("GenuineIntel" 686-class) 647 MHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE
real mem = 519602176 (495MB)
avail mem = 494592000 (471MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 11/30/04, BIOS32 rev. 0 @
0xfb470, SMBIOS rev. 2.2 @ 0xf0800 (38 entries)
bios0: vendor Phoenix Technologies, LTD version "6.00 PG" date 11/30/2004
bios0: VIA Technologies, Inc. CLE266-8233/5
[EMAIL PROTECTED]:etc]# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208
groups: lo
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff000000
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:10:f3:0d:32:6d
description: OOBM
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet6 fe80::210:f3ff:fe0d:326d%fxp0 prefixlen 64 scopeid 0x1
inet 172.16.70.31 netmask 0xffffff00 broadcast 172.16.70.255
fxp1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:10:f3:0d:32:6e
media: Ethernet autoselect (none)
status: no carrier
fxp2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:10:f3:0d:32:6f
description: to white/fxp3
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet6 fe80::210:f3ff:fe0d:326f%fxp2 prefixlen 64 scopeid 0x3
inet 10.1.1.31 netmask 0xffffff00 broadcast 10.1.1.255
fxp3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:10:f3:0d:32:70
description: to orange/eth1
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet6 fe80::210:f3ff:fe0d:3270%fxp3 prefixlen 64 scopeid 0x4
inet 10.2.2.31 netmask 0xffffff00 broadcast 10.2.2.255
enc0: flags=0<> mtu 1536
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33208
groups: pflog
[EMAIL PROTECTED]:etc]#
[EMAIL PROTECTED]:etc]# cat /etc/hoststated.conf
# Macros
# proxies listen on
yellow="10.2.2.31"
# and relay traffic to VPN Tunnel endpoint
white="10.1.1.30"
sshHost="10.2.90.3"
dnsHost="10.2.90.1"
x11Host="10.2.90.3"
# Global Options
log updates
prefork 1
interval 10 # run check every N seconds
timeout 4000 # global timeout in milliseconds for checks
# Protocols
protocol sshProto {
# port 22
protocol tcp
tcp {nodelay, sack, socket buffer 65536, backlog 10}
}
protocol dnsProto {
# port 53
protocol dns
}
protocol nfOracleProto {
# port 1521
protocol tcp
tcp {nodelay, sack, socket buffer 65536, backlog 10}
}
protocol x11Proto {
# port 6000
protocol tcp
tcp {nodelay, sack, socket buffer 65536, backlog 10}
}
# Relays
relay sshRelay {
listen on $yellow port 22
protocol sshProto
forward to $white port 22
#forward to $sshHost port 22
timeout 600
}
relay dnsRelay {
listen on $yellow port 53
protocol dnsProto
forward to $white port 53
#forward to $dnsHost port 53
timeout 60
}
relay nfOracleRelay {
listen on $yellow port 1521
protocol nfOracleProto
forward to $white port 1521
#forward to $ospHost port 1521
timeout 3600
}
relay x11Relay {
listen on $yellow port 6000
protocol x11Proto
forward to $white port 6000
#forward to $x11Host port 6000
timeout 600
}
[EMAIL PROTECTED]:etc]#
[EMAIL PROTECTED]:etc]# hoststated -v -d
startup
init_filter: filter init done
init_tables: created 0 tables
relay_init: adding relay x11Relay
protocol 3: name x11Proto
flags: 0x0004
type: tcp
relay_init: adding relay nfOracleRelay
protocol 2: name nfOracleProto
flags: 0x0004
type: tcp
relay_init: adding relay dnsRelay
protocol 1: name dnsProto
flags: 0x0004
type: dns
relay_init: adding relay sshRelay
protocol 0: name sshProto
flags: 0x0004
type: tcp
relay_launch: running relay x11Relay
relay_launch: running relay nfOracleRelay
relay_launch: running relay dnsRelay
relay_launch: running relay sshRelay
---
A) DNS/UDP Example
Output of "hoststated -v -d" after issuing a DNS lookup on "orange":
relay_dns_log: session 1: request id 0xf4cc flags 0x1:0x0 qd 1 an 0 ns 0 ar 0
relay dnsRelay, session 1 (1 active), 10.2.2.32 -> 10.1.1.30:53, udp timeout
relay_dns_log: session 2: request id 0xf4cc flags 0x1:0x0 qd 1 an 0 ns 0 ar 0
relay dnsRelay, session 2 (1 active), 10.2.2.32 -> 10.1.1.30:53, udp timeout
hostated listens on the right NIC fxp3:
[EMAIL PROTECTED]:root]# tcpdump -i fxp3 -n
tcpdump: listening on fxp3, link-type EN10MB
15:51:39.635373 10.2.2.32.32768 > 10.2.2.31.53: 51934+ A? orange. (24) (DF)
15:51:44.636459 10.2.2.32.32768 > 10.2.2.31.53: 51934+ A? orange. (24) (DF)
hostated passes on the proxied requets to the left NIC fxp2, using the
unexpected/wrong(?) source address of (fxp3)=10.2.2.31, instead of
(fxp2)=10.1.1.31 as in the TCP example below:
[EMAIL PROTECTED]:root]# tcpdump -i fxp2 -n
tcpdump: listening on fxp2, link-type EN10MB
15:42:13.565810 10.2.2.31.53 > 10.1.1.30.53: 5744+ A? orange. (24)
15:42:18.566692 10.2.2.31.53 > 10.1.1.30.53: 6135+ A? orange. (24)
---
B) TCP Example
Output of "hoststated -v -d" after "orange" opens and immediately
closes again an X11 window on a remote X server to the left of
"white":
relay x11Relay, session 5 (1 active), 10.2.2.32 -> 10.1.1.30:6000, done
hostated listens on the right NIC fxp3:
[EMAIL PROTECTED]:root]# tcpdump -i fxp3 -n
tcpdump: listening on fxp3, link-type EN10MB
15:49:36.359944 10.2.2.32.32770 > 10.2.2.31.6000: S
18518406:18518406(0) win 5840 <mss 1460,sackOK,timestamp 74716745
0,nop,wscale 2> (DF) [tos 0x10]
15:49:36.360083 10.2.2.31.6000 > 10.2.2.32.32770: S
2569303658:2569303658(0) ack 18518407 win 65535 <mss
1460,nop,nop,sackOK,nop,wscale 1,nop,nop,timestamp 2174965381
74716745> (DF)
15:49:36.360975 10.2.2.32.32770 > 10.2.2.31.6000: . ack 1 win 1460
<nop,nop,timestamp 74716746 2174965381> (DF) [tos 0x10]
15:49:39.487031 10.2.2.32.32770 > 10.2.2.31.6000: P 1:3(2) ack 1 win
1460 <nop,nop,timestamp 74719873 2174965381> (DF) [tos 0x10]
15:49:39.684656 10.2.2.31.6000 > 10.2.2.32.32770: . ack 3 win 33304
<nop,nop,timestamp 2174965388 74719873> (DF)
15:49:43.873208 10.2.2.32.32770 > 10.2.2.31.6000: F 3:3(0) ack 1 win
1460 <nop,nop,timestamp 74724259 2174965388> (DF) [tos 0x10]
15:49:43.873284 10.2.2.31.6000 > 10.2.2.32.32770: . ack 4 win 33304
<nop,nop,timestamp 2174965396 74724259> (DF)
15:49:43.873720 10.2.2.31.6000 > 10.2.2.32.32770: F 1:1(0) ack 4 win
33304 <nop,nop,timestamp 2174965396 74724259> (DF)
15:49:43.873928 10.2.2.32.1024 > 10.2.2.31.6000: . ack 2569303660 win
1460 <nop,nop,timestamp 74724260 2174965396> (DF)
15:49:45.365551 10.2.2.31.6000 > 10.2.2.32.32770: F 1:1(0) ack 4 win
33304 <nop,nop,timestamp 2174965399 74724259> (DF)
15:49:45.366449 10.2.2.32.1024 > 10.2.2.31.6000: . ack 1 win 1460
<nop,nop,timestamp 74725753 2174965396> (DF)
15:49:48.366031 10.2.2.31.6000 > 10.2.2.32.32770: F 1:1(0) ack 4 win
33304 <nop,nop,timestamp 2174965405 74724259> (DF)
15:49:48.366837 10.2.2.32.1024 > 10.2.2.31.6000: . ack 1 win 1460
<nop,nop,timestamp 74728753 2174965396> (DF)
hostated passes on the proxied requets to the left NIC fxp2, using the
source address (fxp2)=10.1.1.31, as expected:
[EMAIL PROTECTED]:root]# tcpdump -i fxp2 -n
tcpdump: listening on fxp2, link-type EN10MB
15:44:10.623784 10.1.1.31.46713 > 10.1.1.30.6000: S
2296881782:2296881782(0) win 65535 <mss 1460,nop,nop,sackOK,nop,wscale
1,nop,nop,timestamp 3412075455 0> (DF)
15:44:10.625018 10.1.1.30.6000 > 10.1.1.31.46713: S
2783995020:2783995020(0) ack 2296881783 win 16384 <mss
1368,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1707132611
3412075455> (DF)
15:44:10.625083 10.1.1.31.46713 > 10.1.1.30.6000: . ack 1 win 32768
<nop,nop,timestamp 3412075455 1707132611> (DF)
15:44:18.336307 10.1.1.31.46713 > 10.1.1.30.6000: P 1:6(5) ack 1 win
32768 <nop,nop,timestamp 3412075471 1707132611> (DF)
15:44:18.529429 10.1.1.30.6000 > 10.1.1.31.46713: . ack 6 win 17628
<nop,nop,timestamp 1707132627 0> (DF)
15:44:40.750801 10.1.1.31.46713 > 10.1.1.30.6000: F 6:6(0) ack 1 win
32768 <nop,nop,timestamp 3412075515 1707132627> (DF)
15:44:40.751809 10.1.1.30.6000 > 10.1.1.31.46713: . ack 7 win 17628
<nop,nop,timestamp 1707132671 0> (DF)
15:44:40.752193 10.1.1.30.6000 > 10.1.1.31.46713: F 1:1(0) ack 7 win
17628 <nop,nop,timestamp 1707132671 0> (DF)
15:44:40.752244 10.1.1.31.46713 > 10.1.1.30.6000: . ack 2 win 32768
<nop,nop,timestamp 3412075515 1707132671> (DF)
