I have seen interesting cases of multipath sometimes doing the wrong thing.
For outbound connections, it works great. I have confirmed that the packets
go out through each connection in a load-balanced fashion.
However, for incoming connections, I see a SYN packet come in on interface
one, and the SYN-ACK goes out on interface two.
My OpenBSD 4.7/i386 box has four interfaces but I'm just using
two for this test -- rl0 and rl1.
My network diagram is shown below. I've replaced my public IP address
for the rl0 interface with "x.x.x.176" and the default route for that
interface with "x.x.x.129".
The rl1 interface is behind a router that has been configured to
forward all traffic to the OpenBSD box.
x.x.x.129 y.y.y.50
| Router
| 192.168.15.1
| |
| |
rl0 rl1
x.x.x.176 192.168.15.2
The OpenBSD box has Apache running on port 80.
# sysctl | grep multipath
net.inet.ip.multipath=1
net.inet6.ip6.multipath=1
I set up the routing with the following route commands:
# route delete default x.x.x.129
# route add -mpath default x.x.x.129
# route add -mpath default 192.168.15.1
# route -n show | head
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default x.x.x.129 UGSP 0 0 - 8 rl0
default 192.168.15.1 UGSP 0 0 - 8 rl1
127/8 127.0.0.1 UGRS 0 0 33200 8 lo0
127.0.0.1 127.0.0.1 UH 1 0 33200 4 lo0
...
When I telnet in to x.x.x.176 on port 80, I see an incoming SYN
packet on rl0, and the SYN-ACK reply going out on rl1, thus
breaking the telnet session.
When I telnet in to y.y.y.50 on port 80, the incoming SYN packet
goes into rl1, and the SYN-ACK reply goes out on rl1, so the
telnet session was successful for this interface.
However, sometimes the behavior switches, when I can connect to
port 80 on rl1, but not on rl0.
This was with the default PF rules that came with OpenBSD 4.7:
# pfctl -sr
pass all flags S/SA keep state
block drop in on ! lo0 proto tcp from any to any port 6000:6010
I also disabled PF and got the same thing. Do I need PF to make
the addresses stick?
Head of dmesg:
OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
