On 2018/09/19 10:00, Tim Jones wrote:
>
> > This feels like it might be an MTU related problem, especially likely
> > if the connection is going via pppoe or a tunnel - you may need "scrub
> > (max-mss ##)".
> >
> > The way Google's TLS server handshake is setup, it fits in pppoe without
> > fragmentation, most other sites do not this.
> >
> > Otherwise try simplifying pf.conf (one change at a time and test):
> > disable syncookies and change "modulate state" to "keep state", maybe
> > also the random-id scrub. ("syncookies always" in PF doesn't make a
> > lot of sense to me except for testing, especially if only allowing
> > inside->outside traffic, I think "adaptive" would be more usual if
> > using this feature).
>
>
>
> Thanks Stuart. These sound possibly more likely than some of the other
> suggestions (e.g. wrong date/time or bad pf rules ... I'm not that silly).
>
> The connection is not going via PPPoE or tunnel. The immediate next hop is
> an OpenBSD based BGP router (where, incidentally I can't replicate this SSL
> issue, but the router is not (yet) running 6.3 either). The OpenBSD router
> box is then plugged into large carrier routers. So it all this is not
> hanging off the end of some random DSL line !
Is there one OpenBSD BGP router or more, and is PF running there too?
(Basically check with tcpdump on various interfaces along the way that
the packets you expect to receive from the TLS server/s you're
connecting to aren't being dropped somewhere - if there are paths
to/from "the internet" going via multiple stateful firewalls you
can have problems with asymmetric traffic if you're not careful).
> The reason I've got "syncookies always" is because there are various internet
> exposed services (e.g. webservers) sitting behind this PF instance, and as
> far as I can gather syncookies is recommended as a good thing (tm) for these
> sort of applications ? This PF instance is very much a majority out->in
> instance.
>
> But at the same time I'm also unclear as to what the impact of syncookies is
> on states ? The man page talks of "continue the connection with synproxy in
> place", which in my mind implies "synproxy state" ?
>
Synproxy is meant to protect the backend server under syn flood, I'm not
an expert on this but AIUI there are often on-host mechanisms like syn
cache so this isn't needed as much these days except for hosts with
particularly vulnerable TCP stacks (though I'd probably use a TCP relay
for those instead as it gives more isolation).
PF's syncookies are instead to protect the PF state table on the firewall.
No state table entry is created until the client has correctly answered
SYN+ACK avoiding PF states from most blind spoofed SYNs. Using them also
requires synproxy (PF already answered the SYN so the source will see
this as "connection open" at that point, the end destination won't see
anything until the client sends a valid SYN+ACK and the firewall commits
resources to it, it's not until that point when the server first sees a
[proxied] syn to bring up the connection).
For monitoring/debugging under normal conditions I don't like the
"accept connection before the server has seen anything" behaviour so
adaptive generally makes sense to me though it's wise to test the
"adverse conditions" behaviour first (and for that, "always" allows
doing so without actually flooding the firewall with SYNs).
Anyway in your current situation I would be cutting down things which
make not-entirely-necessary changes to packets one by one until the
cause of the problem shows itself..
