23.09.2024 17:48, Stuart Henderson пишет:
On 2024-09-23, kasak <ka...@kasakoff.net> wrote:
23.09.2024 15:22, Brian Conway пишет:
On Mon, Sep 23, 2024, at 6:19 AM, kasak wrote:
Hello, misc!
Could you please share your wisdom about this problem.
On my openbsd firewall, sometimes network become slow and some daemons
stop working.
/var/log/messages have this messages when slowdown is in place:
Sep 23 13:49:34 gater ntpd[30891]: sendto: Permission denied
Sep 23 13:56:22 gater isakmpd[64631]: sendmsg (14, 0x784ce63ce408, 0):
Permission denied
also nginx have this messages:
connect() to 172.16.0.80:443 failed (13: Permission denied) while
connecting to upstream
also i cannot ping nor nslookup anything also because "permission denied"
I found workaround by flushing pf states. After pfctl -F states
everything start to work again.
But maybe i should tune something i did not know about?
How can I diagnose this failures?
You may have a full state table. Try:
pfctl -si
pfctl -ss
Do I understand correctly that "current entries" (pfctl -si) is the
number of states?
Yes but just show all of pfctl -si as that may give other clues too.
pfctl -sm and pfctl -st may also be useful.
Alternatively `pfctl -sa` includes all. If you have run out of available state
tracking, I would spot check what is using up all the state entries and whether
it is expected prior to increasing the limit.
(pfctl -sa will be pretty huge if you have a full state table and you
probably don't want to send that to the list)
Unfortunately, or maybe this is fortunately? I have not experienced any
slowdowns this week.
Nothing surprising here, I have cached this condition only twice for 10+
years of using OpenBSD as a gateway.
But this "twice" happened last week.
But I think the guess was correct. I have been monitoring "current
entries" through all peak hours, and the maximum value was near to 70 000.
Here is output of pfctl, but this is output of normal condition, when
everything is working. Just in case,if I catch any slowdown in future,
so we can compare.
doas pfctl -si
Status: Enabled for 9 days 20:02:33 Debug: err
State Table Total Rate
current entries 61986
half-open tcp 318
searches 13376986308 15742.2/s
inserts 79225635 93.2/s
removals 79163649 93.2/s
Counters
match 97348919 114.6/s
bad-offset 0 0.0/s
fragment 333 0.0/s
short 26083 0.0/s
normalize 223 0.0/s
memory 749907 0.9/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 148233 0.2/s
proto-cksum 16 0.0/s
state-mismatch 642701 0.8/s
state-insert 0 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
translate 0 0.0/s
no-route
kasak@OpenBSD-gater:~$ doas pfctl -sm
states hard limit 100000
src-nodes hard limit 10000
frags hard limit 65536
tables hard limit 1000
table-entries hard limit 200000
pktdelay-pkts hard limit 10000
anchors hard limit 512
doas pfctl -st
tcp.first 120s
tcp.opening 30s
tcp.established 86400s
tcp.closing 900s
tcp.finwait 45s
tcp.closed 90s
tcp.tsdiff 30s
udp.first 60s
udp.single 30s
udp.multiple 60s
icmp.first 20s
icmp.error 10s
other.first 60s
other.single 30s
other.multiple 60s
frag 60s
interval 10s
adaptive.start 60000 states
adaptive.end 120000 states
src.track 0s
