https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209471
Bug ID: 209471
Summary: Listen queue overflow due to too many sockets stuck in
CLOSED state
Product: Base System
Version: 10.3-RELEASE
Hardware: amd64
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: freebsd-b...@freebsd.org
Reporter: rblay...@inoc.net
CC: freebsd-amd64@FreeBSD.org
CC: freebsd-amd64@FreeBSD.org
10.3-RELEASE FreeBSD 10.3-RELEASE #0 r297856M
We are randomly getting daemon applications on a mail server (Dovecot and Exim)
where we start getting listen queue overflows due to hundreds or thousands of
TCP connections stuck in a CLOSED state.
Kernel messages:
sonewconn: pcb 0xfffff800155a3498: Listen queue overflow: 301 already in queue
awaiting acceptance (50 occurrences)
sonewconn: pcb 0xfffff800155a3498: Listen queue overflow: 301 already in queue
awaiting acceptance (50 occurrences)
sonewconn: pcb 0xfffff800155a3498: Listen queue overflow: 301 already in queue
awaiting acceptance (50 occurrences)
sonewconn: pcb 0xfffff800155a3498: Listen queue overflow: 301 already in queue
awaiting acceptance (48 occurrences)
sonewconn: pcb 0xfffff800155a3498: Listen queue overflow: 301 already in queue
awaiting acceptance (50 occurrences)
sonewconn: pcb 0xfffff800155a3498: Listen queue overflow: 301 already in queue
awaiting acceptance (50 occurrences)
...
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.19266 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.12342 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.29123 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.23215 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.56331 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.52066 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.33798 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.34610 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.15283 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.51922 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.7406 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.41955 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.56028 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.6446 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.2474 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.51723 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.51069 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.18158 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.38435 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.46607 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.33359 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.62935 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.11673 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.51459 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.36490 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.27831 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.44081 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.28384 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.43745 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.64070 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.35722 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.63738 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.14573 CLOSED
tcp6 32 0 2607:f058:110:2:.4190 2607:f058:110:2:.12311 CLOSED
...
(hundreds and hundreds of these lines removed)
Looking at sockstat these connections do not seem to be related to the process
anymore...
? ? ? ? tcp6 2607:f058:110:2::1:2:25
2607:f058:110:2::f:0:49398
? ? ? ? tcp6 2607:f058:110:2::1:2:110
2607:f058:110:2::f:0:28079
? ? ? ? tcp6 2607:f058:110:2::1:2:25
2607:f058:110:2::f:1:52383
? ? ? ? tcp6 2607:f058:110:2::1:2:110
2607:f058:110:2::f:1:35856
? ? ? ? tcp6 2607:f058:110:2::1:2:143
2607:f058:110:2::f:0:27734
? ? ? ? tcp6 2607:f058:110:2::1:2:143
2607:f058:110:2::f:1:36851
? ? ? ? tcp6 2607:f058:110:2::1:2:25
2607:f058:110:2::f:0:40977
? ? ? ? tcp6 2607:f058:110:2::1:2:110
2607:f058:110:2::f:0:51172
? ? ? ? tcp6 2607:f058:110:2::1:2:25
2607:f058:110:2::f:1:16197
? ? ? ? tcp6 2607:f058:110:2::1:2:110
2607:f058:110:2::f:1:1999
? ? ? ? tcp6 2607:f058:110:2::1:2:143
2607:f058:110:2::f:0:60423
? ? ? ? tcp6 2607:f058:110:2::1:2:143
2607:f058:110:2::f:1:16527
? ? ? ? tcp6 2607:f058:110:2::1:2:25
2607:f058:110:2::f:0:34327
? ? ? ? tcp6 2607:f058:110:2::1:2:110
2607:f058:110:2::f:0:5437
? ? ? ? tcp6 2607:f058:110:2::1:2:25
2607:f058:110:2::f:1:30114
? ? ? ? tcp6 2607:f058:110:2::1:2:110
2607:f058:110:2::f:1:57136
? ? ? ? tcp6 2607:f058:110:2::1:2:143
2607:f058:110:2::f:0:58399
? ? ? ? tcp6 2607:f058:110:2::1:2:143
2607:f058:110:2::f:1:37073
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:11673
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:33798
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:65207
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:13326
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:27879
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:2899
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:39172
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:19330
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:18694
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:1251
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:43392
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:44343
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:36523
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:41551
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:24288
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:3830
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:43978
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:8897
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:65187
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:14214
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:1:55279
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:31178
? ? ? ? tcp6 2607:f058:110:2::1:2:4190
2607:f058:110:2::f:0:49242
...
(hundreds or thousands of lines removed)
The only way to fix the issue is to reboot the server. (in this case a VMware
ESXi 5.5 VM)
Network driver is "vmx" if that makes any difference.
--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
freebsd-amd64@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-amd64
To unsubscribe, send any mail to "freebsd-amd64-unsubscr...@freebsd.org"
