Hi, haven't read your original email but if my assumptions about your setup are correct is the VPN tunnel dropping every now and then?
I had a similar issue with 4 OBSD firewalls (2 at each end), all running isakmpd and sasyncd to keep the SAs in sync between a pair. With the tunnels explicitly set to use the CARP IPs as endpoints. You need the static route to point to the internal interface to make sure that packets generated by the firewall itself have a source IP set to the internal net thus allowing the IPSec policy route to be used (as it defines both the source and dest net, not just the dest net like a normal route). However whilst pinging the remote net from the live firewall works, I found that if you try and ping the remote net from the backup firewall, the packet does *not* get routed to the CARP master (why would it!). The IPSec route is there.. The backup firewall encaps the packet and sends it out its own external interface using the dets from sasyncd, but the master firewall at the other end will see the packet as coming from 'another' box with the same details but not the full correct policy and so the tunnel is blocked for a period until the master renegotiates it. We had to modify all our monitoring scripts to not 'phone home' if the box is a backup. Only the master firewall can use the VPN. I also submitted some suggested modifications to /etc/rc.d/sasyncd and /etc/rc.d/isakmpd here in the past which makes the setup and failover of VPNs much faster and more stable. Can dig out if you can't find them. Cheers, Andy Sent from my iPhone > On 6 Mar 2014, at 19:00, Zach Leslie <xaque...@gmail.com> wrote: > > On Wed, Mar 05, 2014 at 11:05:11PM -0600, Amit Kulkarni wrote: >>> If PF information is needed, I can provide and obscure, but I didn't >>> expect it to be >>> the issue. >> >> i am no expert on this. but if it is a packet loss issue, you need to post >> the obscured pf.conf > > Fair point. I've not seen any related dropped packets due to PF with > tcpdump -nei pflog0, so I didn't think it would be related to PF. > > Not line-wrapped for readability. > > match out on em0 from ! (em0:network) to any nat-to (em0:0) > block drop in log all > pass out all flags S/SA > pass out on em0 proto tcp all flags S/SA modulate state > pass in proto icmp from <routed_networks> to <routed_networks> > pass in proto ipv6-icmp from <routed_networks> to <routed_networks> > pass in from <routed_networks> to ! <routed_networks> flags S/SA > pass in from <routed_networks> to any flags S/SA > pass in proto udp from <routed_networks> to <dns_servers> port = 53 > pass out on em0 proto udp all > pass out on em0 proto icmp all > pass on em0 inet proto carp all > pass on em0 proto icmp from any to (em0:network) > pass on em1 inet proto pfsync all > pass in on em0 inet proto udp from 66.77.88.10 to 1.2.3.5 port = 500 > pass in on em0 inet proto udp from 66.77.88.10 to 1.2.3.5 port = 4500 > pass in on em0 inet proto esp from 66.77.88.10 to 1.2.3.5 > pass in on enc0 inet proto ipencap from 66.77.88.10 to 1.2.3.5 keep state > (if-bound) > pass out on em0 inet proto udp from 1.2.3.5 to 66.77.88.10 port = 500 > pass out on em0 inet proto udp from 1.2.3.5 to 66.77.88.10 port = 4500 > pass out on em0 inet proto esp from 1.2.3.5 to 66.77.88.10 > pass out on enc0 inet proto ipencap from 1.2.3.5 to 66.77.88.10 keep state > (if-bound) > pass out on enc0 inet from 1.2.3.5 to <pdx_nets> flags S/SA keep state > (if-bound) > pass out on enc0 from <opdx_nets> to <pdx_nets> flags S/SA keep state > (if-bound) > pass in on enc0 from <pdx_nets> to <opdx_nets> flags S/SA keep state > (if-bound) > pass in on enc0 inet from 66.77.88.10 to <opdx_nets> flags S/SA keep state > (if-bound) > pass in on em0 inet proto tcp from 66.77.88.17 to any port = 22 flags S/SA > pass in on em0 inet proto tcp from 1.2.3.7 to 1.2.3.6 port = 500 flags S/SA > > This morning as a test, I've disabled isakmpd sync feature, and shutdown > sasycnd on all firewalls, as well as isakmpd on the secondaries at each > location and the connection seems to be much improved. I've not lost > any connections in the last 4 hours which is much improved. > > Not sure if sasyncd is actually causing the issue, but disabling it > to gain an improved connections certainly doesn't seem great from an HA > standpoint. > > I've also got a couple static routes in the inet table that point the > remote network to the internal carp address for routing purposes. This > allows the traffic generated by the secondary firewall to reach the > remote network due to the fact that the secondary does not hold the > master status of the carp, and therfore can't use the IPSec directly. > > I do wonder though, since I also have a flow for the same network, the > encap and inet routing table have a route for the same network. Which > takes priority? Just something to point out since it could be causing > troubles. > > Regards, > > -- > Zach
