Hello,
I'm testing new servers (same mboard/cpu) and getting 'congestion' in
the pf information output on very low level of traffic (~100 pps).
Traffic is coming in on em0:901 and em0:2906 using carp.
Traffic level is monitored on the switch port.
First tried 6.5-stable and now using 6.6-beta. Only default daemons
and nsd are running.
Can somebody help me figure this out ?
Thanks !
Eric Camirand
===
sysctl kern.netlivelocks
kern.netlivelocks=2
===
Status: Enabled for 0 days 19:25:59 Debug: info
Hostid: 0xdc985578
Checksum: 0x5b36311462da2ab76b362f191125df49
Interface Stats for egress IPv4 IPv6
Bytes In 410275731 159715549
Bytes Out 645392204 219562746
Packets In
Passed 5230566 1821014
Blocked 15434 298
Packets Out
Passed 4496816 1353481
Blocked 1 22
State Table Total Rate
current entries 2507
half-open tcp 0
searches 12917617 184.6/s
inserts 5637553 80.6/s
removals 5635046 80.5/s
Source Tracking Table
current entries 0
searches 0 0.0/s
inserts 0 0.0/s
removals 0 0.0/s
Counters
match 5653419 80.8/s
bad-offset 0 0.0/s
fragment 0 0.0/s
short 0 0.0/s
normalize 15 0.0/s
memory 0 0.0/s
bad-timestamp 0 0.0/s
congestion 5 0.0/s
ip-option 116 0.0/s
proto-cksum 0 0.0/s
state-mismatch 13 0.0/s
state-insert 0 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
translate 0 0.0/s
no-route 0 0.0/s
Limit Counters
max states per rule 0 0.0/s
max-src-states 0 0.0/s
max-src-nodes 0 0.0/s
max-src-conn 0 0.0/s
max-src-conn-rate 0 0.0/s
overload table insertion 0 0.0/s
overload flush states 0 0.0/s
synfloods detected 0 0.0/s
syncookies sent 0 0.0/s
syncookies validated 0 0.0/s
Adaptive Syncookies Watermarks
start 25000 states
end 12500 states
===
@0 block drop log all
[ Evaluations: 5652928 Packets: 0 Bytes: 0 States: 0 ]
[ Inserted: uid 0 pid 62066 State Creations: 0 ]
@1 pass quick proto carp all
[ Evaluations: 5652928 Packets: 105212 Bytes: 6943972 States: 2 ]
[ Inserted: uid 0 pid 62066 State Creations: 2 ]
@2 block drop in quick on egress from <BAD:0> to any
[ Evaluations: 5652926 Packets: 0 Bytes: 0 States: 0 ]
[ Inserted: uid 0 pid 62066 State Creations: 0 ]
@3 pass in quick on egress proto tcp from any to <DNS:8> port = 53 flags S/SA
[ Evaluations: 5646195 Packets: 141995 Bytes: 7940619 States: 52 ]
[ Inserted: uid 0 pid 62066 State Creations: 21188 ]
@4 pass in quick on egress proto tcp from <MGMT:1> to any port = 22 flags S/SA
[ Evaluations: 13461 Packets: 7958 Bytes: 5424896 States: 1 ]
[ Inserted: uid 0 pid 62066 State Creations: 14 ]
@5 pass in quick on egress proto udp from any to <DNS:8> port = 53
[ Evaluations: 5624993 Packets: 11196223 Bytes: 1279868426 States: 3051 ]
[ Inserted: uid 0 pid 62066 State Creations: 5582822]
@6 pass in quick on egress inet proto icmp all
[ Evaluations: 42171 Packets: 11503 Bytes: 727676 States: 2 ]
[ Inserted: uid 0 pid 62066 State Creations: 5041 ]
@7 pass in quick on egress inet6 proto ipv6-icmp all
[ Evaluations: 36988 Packets: 59374 Bytes: 4213306 States: 4 ]
[ Inserted: uid 0 pid 62066 State Creations: 21286 ]
@8 block drop in log quick on egress all
[ Evaluations: 15604 Packets: 15604 Bytes: 1183076 States: 0 ]
[ Inserted: uid 0 pid 62066 State Creations: 0 ]
@9 pass out quick on egress all flags S/SA
[ Evaluations: 6731 Packets: 24872 Bytes: 9727811 States: 2 ]
[ Inserted: uid 0 pid 62066 State Creations: 6729 ]
@10 block drop out quick on egress all
[ Evaluations: 2 Packets: 2 Bytes: 144 States: 0 ]
[ Inserted: uid 0 pid 62066 State Creations: 0 ]
===
1 users Load 0.04 0.05 0.00 so-1-s-010 22:22:21
IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM
System 0 256 47 16
2112 38 12
lo0
em0 2050 38 10 256 38
em1
em2
em3
enc0
vlan901
pflog0
vlan2906
carp0
carp1
carp2
carp3
===
OpenBSD 6.6-beta (GENERIC.MP) #205: Mon Aug 12 21:58:39 MDT 2019
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34156728320 (32574MB)
avail mem = 33108889600 (31575MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x7fb76000 (68 entries)
bios0: vendor American Megatrends Inc. version "2.2" date 05/23/2018
bios0: Supermicro X11SSH-F
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG HPET LPIT SSDT SSDT SSDT DBGP
DBG2 SSDT PRAD SSDT UEFI SSDT DMAR EINJ ERST BERT HEST
acpi0: wakeup devices PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4) PEGP(S4)
RP09(S4) PXSX(S4) RP10(S4) PXSX(S4) RP11(S4) PXSX(S4) RP12(S4) PXSX(S4)
RP13(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1240 v6 @ 3.70GHz, 3701.52 MHz, 06-9e-09
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
tsc_timecounter_init: TSC skew=0 observed drift=0
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 24MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
TSC skew=-2
cpu1: Intel(R) Xeon(R) CPU E3-1240 v6 @ 3.70GHz, 3700.01 MHz, 06-9e-09
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
tsc_timecounter_init: TSC skew=-2 observed drift=0
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
TSC skew=-3
cpu2: Intel(R) Xeon(R) CPU E3-1240 v6 @ 3.70GHz, 3700.00 MHz, 06-9e-09
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
tsc_timecounter_init: TSC skew=-3 observed drift=0
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
TSC skew=1
cpu3: Intel(R) Xeon(R) CPU E3-1240 v6 @ 3.70GHz, 3700.00 MHz, 06-9e-09
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
tsc_timecounter_init: TSC skew=1 observed drift=0
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 23999999 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEG0)
acpiprt2 at acpi0: bus -1 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus -1 (RP09)
acpiprt5 at acpi0: bus -1 (RP13)
acpiprt6 at acpi0: bus 2 (RP01)
acpiprt7 at acpi0: bus 3 (RP02)
acpiprt8 at acpi0: bus -1 (RP03)
acpiprt9 at acpi0: bus -1 (RP04)
acpiprt10 at acpi0: bus -1 (RP05)
acpiprt11 at acpi0: bus -1 (RP06)
acpiprt12 at acpi0: bus 4 (RP07)
acpiprt13 at acpi0: bus 5 (BR53)
acpiprt14 at acpi0: bus -1 (RP08)
acpiprt15 at acpi0: bus -1 (RP17)
acpiprt16 at acpi0: bus -1 (RP18)
acpiprt17 at acpi0: bus -1 (RP19)
acpiprt18 at acpi0: bus -1 (RP20)
acpiprt19 at acpi0: bus -1 (RP14)
acpiprt20 at acpi0: bus -1 (RP15)
acpiprt21 at acpi0: bus -1 (RP16)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C3(200@256 mwait.1@0x40), C2(200@151 mwait.1@0x33),
C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C3(200@256 mwait.1@0x40), C2(200@151 mwait.1@0x33),
C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C3(200@256 mwait.1@0x40), C2(200@151 mwait.1@0x33),
C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C3(200@256 mwait.1@0x40), C2(200@151 mwait.1@0x33),
C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: PG00, resource for PEG0
acpipwrres1 at acpi0: PG01, resource for PEG1
acpipwrres2 at acpi0: PG02, resource for PEG2
acpipwrres3 at acpi0: WRST
acpipwrres4 at acpi0: WRST
acpipwrres5 at acpi0: WRST
acpipwrres6 at acpi0: WRST
acpipwrres7 at acpi0: WRST
acpipwrres8 at acpi0: WRST
acpipwrres9 at acpi0: WRST
acpipwrres10 at acpi0: WRST
acpipwrres11 at acpi0: WRST
acpipwrres12 at acpi0: WRST
acpipwrres13 at acpi0: WRST
acpipwrres14 at acpi0: WRST
acpipwrres15 at acpi0: WRST
acpipwrres16 at acpi0: WRST
acpipwrres17 at acpi0: WRST
acpipwrres18 at acpi0: WRST
acpipwrres19 at acpi0: WRST
acpipwrres20 at acpi0: WRST
acpipwrres21 at acpi0: WRST
acpipwrres22 at acpi0: WRST
acpipwrres23 at acpi0: FN00, resource for FAN0
acpipwrres24 at acpi0: FN01, resource for FAN1
acpipwrres25 at acpi0: FN02, resource for FAN2
acpipwrres26 at acpi0: FN03, resource for FAN3
acpipwrres27 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 119 degC
acpitz1 at acpi0: critical temperature is 119 degC
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
acpicmos0 at acpi0
"IPI0001" at acpi0 not configured
"INT0E0C" at acpi0 not configured
acpibtn0 at acpi0: SLPB
"INT33A1" at acpi0 not configured
acpibtn1 at acpi0: PWRB
"ACPI000D" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
ipmi at mainbus0 not configured
cpu0: using Skylake AVX MDS workaround
cpu0: Enhanced SpeedStep 3701 MHz: speeds: 3701, 3700, 3500, 3300, 3100, 2900,
2700, 2500, 2200, 2000, 1800, 1600, 1400, 1200, 1000, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v6/7 Host" rev 0x05
ppb0 at pci0 dev 1 function 0 "Intel Core 6G PCIE" rev 0x05: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 2 int 16, address
00:15:17:a6:52:fa
em1 at pci1 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 2 int 17, address
00:15:17:a6:52:fb
"Intel 100 Series ISH" rev 0x31 at pci0 dev 19 function 0 not configured
xhci0 at pci0 dev 20 function 0 "Intel 100 Series xHCI" rev 0x31: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00
addr 1
pchtemp0 at pci0 dev 20 function 2 "Intel 100 Series Thermal" rev 0x31
"Intel 100 Series MEI" rev 0x31 at pci0 dev 22 function 0 not configured
"Intel 100 Series MEI" rev 0x31 at pci0 dev 22 function 1 not configured
ahci0 at pci0 dev 23 function 0 "Intel 100 Series AHCI" rev 0x31: msi, AHCI
1.3.1
ahci0: port 4: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 4 lun 0: <ATA, INTEL SSDSC2BW24, 400i> SCSI3 0/direct
fixed naa.5001517bb2af180f
sd0: 228936MB, 512 bytes/sector, 468862128 sectors, thin
ppb1 at pci0 dev 28 function 0 "Intel 100 Series PCIE" rev 0xf1: msi
pci2 at ppb1 bus 2
em2 at pci2 dev 0 function 0 "Intel I210" rev 0x03: msi, address
ac:1f:6b:77:ed:c6
ppb2 at pci0 dev 28 function 1 "Intel 100 Series PCIE" rev 0xf1: msi
pci3 at ppb2 bus 3
em3 at pci3 dev 0 function 0 "Intel I210" rev 0x03: msi, address
ac:1f:6b:77:ed:c7
ppb3 at pci0 dev 28 function 6 "Intel 100 Series PCIE" rev 0xf1: msi
pci4 at ppb3 bus 4
ppb4 at pci4 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
pci5 at ppb4 bus 5
vga1 at pci5 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel C236 LPC" rev 0x31
"Intel 100 Series PMC" rev 0x31 at pci0 dev 31 function 2 not configured
ichiic0 at pci0 dev 31 function 4 "Intel 100 Series SMBus" rev 0x31: apic 2 int
16
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: stts2004
sdtemp1 at iic0 addr 0x19: stts2004
sdtemp2 at iic0 addr 0x1a: stts2004
sdtemp3 at iic0 addr 0x1b: stts2004
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
uhub1 at uhub0 port 14 configuration 1 interface 0 "ATEN International product
0x7000" rev 2.00/0.00 addr 2
uhidev0 at uhub1 port 1 configuration 1 interface 0 "ATEN International product
0x2419" rev 1.10/1.00 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub1 port 1 configuration 1 interface 1 "ATEN International product
0x2419" rev 1.10/1.00 addr 3
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (2b8c5fc37393b73b.a) swap on sd0b dump on sd0b
