On Sat, Mar 17, 2018 at 02:34:09PM +1100, Cameron Simpson wrote:
> I'm running OpenBSD 6.0 i386 on a Soekris as my local firewall. We had/have
> a problem with network dropouts on our NBN satellite connection which I
> believe I've traced to the firewall's ARP entry for the upstream gateway
> expiring.
>
> The problem appears to be that once the ARP entry expires, the firewall does
> not issue an ARP who-was request to renew the entry. As a consequence
> packets can't be forwarded to the gateway and it looks like an ISP outage.
> This state persists for periods of up to 10 minutes.
>
> During the "outage" DHCP on the ISP link works (presumably because that
> doesn't involve the arp table) but pings to the gateway do not, and nor does
> any other normal IP traffic which requires using the gateway.
>
> I left a tcpdump running for the gateway host IP and noticed this morning
> that immediately after an ARP request occurred and was answered
> (immediately) that traffic commenced working again, which led to to pursuing
> this.
>
> I don't understand why, since the gateway address doesn't have a current ARP
> entry, the firewall does not imemdiately issue an ARP request for it. Even a
> ping directly from the firewall to the gateway address does not cause an ARP
> request.
>
> In case it is relevant, all the through traffic is directed via PF nat-to
> rules, but I suspect this isn't related because direct ping traffic from the
> firewall also doesn't work. On the other hand, there's a secondary interface
> to a 3G modem which doesn't do this, and traffic through that interface is
> not NATed because the 3G modem does it.
>
> Finally, I've done the following to verify the issue:
>
> Waited for the ARP entry to expire, and saw throughput cease and direct
> pings of the gateway from the firewall fail:
>
> ping 172.16.20.254
> PING 172.16.20.254 (172.16.20.254): 56 data bytes
> ping: sendto: Host is down
> ping: wrote 172.16.20.254 64 chars, ret=-1
> ping: sendto: Host is down
> ping: wrote 172.16.20.254 64 chars, ret=-1
>
> I added the ARP entry by hand with the arp command and throughput and pings
> resumed immediately.
>
> I've manually removed the ARP entry and seem identical symptoms, and I've
> manually added a static ARP entry for the gateway and the connection has
> been solid for several hours now. Versus "outages" every hour, if not more
> frequently.
>
> I would like to understand this behaviour and to know if it is, as it
> appears, a bug.
>
> Cheers,
> Cameron Simpson <c...@cskk.id.au>
1. OpenBSD 6.0 is not suppored anymore. Upgrade, it's easy almost always.
2. You really should add details about your network config. Otherwise,
we can only guess.
-Otto
