I'm running a pair of unbound resolvers and am attempting to optimize
performance on them. This stemmed from noticing a couple of issues in the logs.
Dec 2 11:26:52 ns1 unbound: [54230:5] error: recvfrom 26 failed: Host is down
Dec 2 11:27:11 ns1 unbound: [54230:5] notice: sendto failed: Resource
temporarily unavailable
Dec 2 11:27:11 ns1 unbound: [54230:5] notice: remote address is 192.168.2.42
port 5088
I believed the first message is related to a dropped UDP request or subsequent
response. 'netstat -p -u udp' shows "dropped due to full socket buffers".
This was significantly reduced by increasing,
net.inet.udp.recvspace=262144
net.inet.udp.sendspace=262144
Unfortunately, I'm still seeing a few UDP drops. Is there a danger in setting
this is high?
ns1$ netstat -s -p udp
udp:
698584369 datagrams received
0 with incomplete header
0 with bad data length field
2508 with bad checksum
676259 with no checksum
86709458 input packets software-checksummed
706308843 output packets software-checksummed
641800 dropped due to no socket
0 broadcast/multicast datagrams dropped due to no socket
0 dropped due to missing IPsec protection
77324 dropped due to full socket buffers
697862737 delivered
706308952 datagrams output
698578008 missed PCB cache
The second log message seems to stem from a dropped TCP request. There seems
to be a significant number of these and I'm assuming they stem from "452447 SYN
packets dropped due to queue or memory full" as the number of log message is in
the same range as the number of dropped SYN packets.
ns1$ netstat -s -p tcp
tcp:
1856161 packets sent
359575 data packets (73608768 bytes)
27022 data packets (5076843 bytes) retransmitted
0 fast retransmitted packets
928517 ack-only packets (414664 delayed)
0 URG only packets
67 window probe packets
2217 window update packets
538808 control packets
271352 packets software-checksummed
2391157 packets received
739060 acks (for 71221089 bytes)
225691 duplicate acks
506 acks for unsent data
0 acks for old data
473441 packets (101441404 bytes) received in-sequence
111074 completely duplicate packets (75769595 bytes)
21701 old duplicate packets
3 packets with some duplicate data (112 bytes duplicated)
231945 out-of-order packets (88494422 bytes)
21 packets (0 bytes) of data after window
0 window probes
34417 window update packets
6771 packets received after close
52 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
0 discarded for missing IPsec protection
0 discarded due to memory shortage
231084 packets software-checksummed
0 bad/missing md5 checksums
0 good md5 checksums
213191 connection requests
156110 connection accepts
340472 connections established (including accepts)
369167 connections closed (including 14600 drops)
0 connections drained
14167 embryonic connections dropped
860911 segments updated rtt (of 838375 attempts)
40788 retransmit timeouts
3005 connections dropped by rexmit timeout
69 persist timeouts
6563 keepalive timeouts
0 keepalive probes sent
0 connections dropped by keepalive
12445 correct ACK header predictions
222843 correct data packet header predictions
828362 PCB cache misses
40214 dropped due to no socket
0 ECN connections accepted
0 ECE packets received
0 CWR packets received
9148 CE packets received
0 ECT packets sent
0 ECE packets sent
0 CWR packets sent
cwr by fastrecovery: 385
cwr by timeout: 40788
cwr by ecn: 0
3161 bad connection attempts
452447 SYN packets dropped due to queue or memory full
161093 SYN cache entries added
0 hash collisions
156110 completed
0 aborted (no space to build PCB)
252 timed out
0 dropped due to overflow
0 dropped due to bucket overflow
4731 dropped due to RST
0 dropped due to ICMP unreachable
2809 SYN,ACKs retransmitted
913 duplicate SYNs received for entries already in the cache
0 SYNs dropped (no route or no space)
2 SYN cache seeds with new random
293 hash bucket array size in current SYN cache
0 entries in current SYN cache, limit is 10255
0 longest bucket length in current SYN cache, limit is 105
38907 uses of current SYN cache left
385 SACK recovery episodes
11 segment rexmits in SACK recovery episodes
3925 byte rexmits in SACK recovery episodes
461 SACK options received
52940 SACK options sent
Based on the unbound docs I've tweaked some setting in unbound.conf.
ns1$ more /var/unbound/etc/unbound.conf
num-threads: 6
msg-cache-slabs: 4
rrset-cache-slabs: 4
infra-cache-slabs: 4
key-cache-slabs: 4
rrset-cache-size: 50m
msg-cache-size: 128m
outgoing-range: 8192
num-queries-per-thread: 4096
outgoing-num-tcp: 20
incoming-num-tcp: 20
Any insight on reducing the TCP "queue or memory full" issues would be
appreciated.
-Steve S.
