Hi misc, On a fresh install of OpenBSD 6.7 + binary patches, unbound appears to rapidly consume system memory when DNS-over-TLS forwarding is enabled.
vmstat -w 300: -------------- procs memory page disks traps cpu r s avm fre flt re pi po fr sr sd0 cd0 int sys cs us sy id .. 0 73 3858M 1826M 41 0 0 0 0 0 0 0 498 2148 1085 1 0 98 0 73 3868M 1816M 39 0 0 0 0 0 0 0 443 1919 956 2 0 98 0 73 3879M 1805M 37 0 0 0 0 0 0 0 477 2076 1026 2 0 98 0 73 3889M 1794M 37 0 0 0 0 0 0 0 462 2012 995 2 0 98 0 73 3900M 1783M 40 0 0 0 0 0 0 0 478 2142 1038 1 0 98 0 73 3912M 1771M 38 0 0 0 0 0 0 0 515 2292 1125 1 0 99 0 73 3925M 1757M 39 0 0 0 0 0 0 0 559 2418 1201 2 0 98 0 73 3938M 1743M 42 0 0 0 0 0 0 0 566 2439 1214 2 0 98 0 72 3949M 1733M 36 0 0 0 0 0 0 0 529 2327 1146 2 0 98 0 72 3961M 1720M 34 0 0 0 0 0 0 0 533 2332 1136 2 0 97 ps output (5 minute interval): ------------------------------ USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND .. _unbound 23318 7.2 43.0 3589196 3596452 ?? S Wed10AM 115:20.67 unbound _unbound 23318 6.4 43.1 3598140 3605396 ?? S Wed10AM 115:42.88 unbound _unbound 23318 4.9 43.2 3608180 3615436 ?? S Wed10AM 116:04.80 unbound _unbound 23318 3.0 43.3 3617712 3624968 ?? S Wed10AM 116:23.78 unbound _unbound 23318 3.3 43.4 3627064 3634320 ?? S Wed10AM 116:39.40 unbound _unbound 23318 2.6 43.5 3637404 3644660 ?? S Wed10AM 116:50.55 unbound _unbound 23318 8.0 43.7 3649500 3656756 ?? S Wed10AM 117:13.39 unbound _unbound 23318 3.8 43.8 3661676 3668952 ?? S Wed10AM 117:36.19 unbound _unbound 23318 4.6 44.0 3672968 3680264 ?? S Wed10AM 117:54.95 unbound _unbound 23318 11.0 44.1 3683992 3691308 ?? S Wed10AM 118:23.15 unbound Flushing the cache does not free up memory: # vmstat procs memory page disks traps cpu r s avm fre flt re pi po fr sr sd0 cd0 int sys cs us sy id 1 71 4112M 1563M 28 0 0 0 0 0 0 0 243 1067 546 1 0 99 # unbound-control reload ok --wait-- # vmstat procs memory page disks traps cpu r s avm fre flt re pi po fr sr sd0 cd0 int sys cs us sy id 1 71 4113M 1560M 28 0 0 0 0 0 0 0 243 1069 547 1 0 99 The number of queries on this server is negligible; 'unbound-control stats' reports between 25-30 queries per second. I've found several posts on non-OpenBSD mailing lists regarding unbound not freeing memory when DoT forwarding is enabled, but those appear to have been fixed some time ago and the fixes have been applied to the OpenBSD source tree. Is anyone else using unbound with DoT turned on and is experiencing this problem? unbound configuration and dmesg below. Any information will be greatly appreciated. Kind regards, Jon /var/unbound/etc/unbound.conf: ------------------------------ server: interface: 0.0.0.0 interface: ::0 access-control: 0.0.0.0/0 refuse access-control: 127.0.0.0/8 allow access-control: 10.0.0.0/8 allow access-control: 172.16.0.0/12 allow access-control: x.x.x.x/24 allow access-control: ::0/0 refuse access-control: ::1 allow hide-identity: yes hide-version: yes root-hints: "/var/unbound/etc/root.hints" auto-trust-anchor-file: "/var/unbound/db/root.key" val-log-level: 2 private-address: 0.0.0.0/8 # "This" network private-address: 10.0.0.0/8 # Private-use networks private-address: 100.64.0.0/10 # Carrier-grade NAT private-address: 127.0.0.0/8 # Loopback addresses to localhost private-address: 169.254.0.0/16 # Link-local addresses private-address: 172.16.0.0/12 # Private-use networks private-address: 192.0.0.0/24 # IANA IPv4 special purpose address registry private-address: 192.0.2.0/24 # Documentation network TEST-NET-1 private-address: 192.168.0.0/16 # Private-use networks private-address: 198.18.0.0/15 # Network inter-connect benchmark testing private-address: 198.51.100.0/24 # Documentation network TEST-NET-2 private-address: 203.0.113.0/24 # Documentation network TEST-NET-3 private-address: 233.252.0.0/24 # Documentation network MCAST-TEST-NET private-address: 255.255.255.255/32 # Limited broadcast private-address: ::1/128 # Loopback addresses to localhost private-address: 2001:db8::/32 # Documentation network IPv6 private-address: fc00::/8 # Unique local address (ULA) part of "fc00::/7", not defined yet private-address: fd00::/8 # Unique local address (ULA) part of "fc00::/7", "/48" prefix group private-address: fe80::/10 # Link-local addresses (LLA) aggressive-nsec: yes local-zone: "foo.com" redirect local-data: "foo.com A x.x.x.x" local-zone: "bar.com" redirect local-data: "bar.com A x.x.x.x" tls-cert-bundle: "/etc/ssl/cert.pem" remote-control: control-enable: yes control-interface: /var/run/unbound.sock forward-zone: name: "." forward-tls-upstream: yes forward-addr: 1.1.1.1@853#one.one.one.one forward-addr: 1.0.0.1@853#one.one.one.one forward-addr: 8.8.8.8@853#dns.google forward-addr: 8.8.4.4@853#dns.google forward-first: yes dmesg: ------ OpenBSD 6.7 (GENERIC.MP) #2: Thu Jun 4 09:55:08 MDT 2020 r...@syspatch-67-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 8573026304 (8175MB) avail mem = 8300580864 (7916MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe0010 (248 entries) bios0: vendor Phoenix Technologies LTD version "6.00" date 12/12/2018 bios0: VMware, Inc. VMware Virtual Platform acpi0 at bios0: ACPI 4.0 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP BOOT APIC MCFG SRAT HPET WAET acpi0: wakeup devices PCI0(S3) USB_(S1) P2P0(S3) S1F0(S3) S2F0(S3) S8F0(S3) S16F(S3) S17F(S3) S18F(S3) S22F(S3) S23F(S3) S24F(S3) S25F(S3) PE40(S3) S1F0(S3) PE50(S3) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) Silver 4208 CPU @ 2.10GHz, 2094.31 MHz, 06-55-07 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,AVX512CD,AVX512BW,AVX512VL,PKU,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,ARAT,XSAVEOPT,XSAVEC,XSAVES cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 65MHz cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Xeon(R) Silver 4208 CPU @ 2.10GHz, 2094.13 MHz, 06-55-07 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,AVX512CD,AVX512BW,AVX512VL,PKU,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,ARAT,XSAVEOPT,XSAVEC,XSAVES cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 0, package 2 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Xeon(R) Silver 4208 CPU @ 2.10GHz, 2094.12 MHz, 06-55-07 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,AVX512CD,AVX512BW,AVX512VL,PKU,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,ARAT,XSAVEOPT,XSAVEC,XSAVES cpu2: 256KB 64b/line 8-way L2 cache cpu2: smt 0, core 0, package 4 cpu3 at mainbus0: apid 6 (application processor) cpu3: Intel(R) Xeon(R) Silver 4208 CPU @ 2.10GHz, 2094.14 MHz, 06-55-07 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,AVX512CD,AVX512BW,AVX512VL,PKU,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,ARAT,XSAVEOPT,XSAVEC,XSAVES cpu3: 256KB 64b/line 8-way L2 cache cpu3: smt 0, core 0, package 6 ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins acpimcfg0 at acpi0 acpimcfg0: addr 0xf0000000, bus 0-127 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpicpu0 at acpi0: C1(@1 halt!) acpicpu1 at acpi0: C1(@1 halt!) acpicpu2 at acpi0: C1(@1 halt!) acpicpu3 at acpi0: C1(@1 halt!) acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 acpicmos0 at acpi0 "PNP0A05" at acpi0 not configured acpiac0 at acpi0: AC unit online pvbus0 at mainbus0: VMware vmt0 at pvbus0 pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82443BX AGP" rev 0x01 ppb0 at pci0 dev 1 function 0 "Intel 82443BX AGP" rev 0x01 pci1 at ppb0 bus 1 pcib0 at pci0 dev 7 function 0 "Intel 82371AB PIIX4 ISA" rev 0x08 pciide0 at pci0 dev 7 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility pciide0: channel 0 disabled (no drives) pciide0: channel 1 disabled (no drives) piixpm0 at pci0 dev 7 function 3 "Intel 82371AB Power" rev 0x08: SMBus disabled "VMware VMCI" rev 0x10 at pci0 dev 7 function 7 not configured vga1 at pci0 dev 15 function 0 "VMware SVGA II" rev 0x00 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) mpi0 at pci0 dev 16 function 0 "Symbios Logic 53c1030" rev 0x01: apic 1 int 17 mpi0: 0, firmware 1.3.41.32 scsibus1 at mpi0: 16 targets, initiator 7 sd0 at scsibus1 targ 0 lun 0: <VMware, Virtual disk, 2.0> sd0: 1048576MB, 512 bytes/sector, 2147483648 sectors mpi0: target 0 Sync at 160MHz width 16bit offset 127 QAS 1 DT 1 IU 1 ppb1 at pci0 dev 17 function 0 "VMware PCI" rev 0x02 pci2 at ppb1 bus 2 uhci0 at pci2 dev 0 function 0 "VMware UHCI" rev 0x00: apic 1 int 18 ehci0 at pci2 dev 1 function 0 "VMware EHCI" rev 0x00: apic 1 int 19 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 configuration 1 interface 0 "VMware EHCI root hub" rev 2.00/1.00 addr 1 ahci0 at pci2 dev 3 function 0 "VMware AHCI" rev 0x00: apic 1 int 17, AHCI 1.3 ahci0: port 0: 6.0Gb/s scsibus2 at ahci0: 32 targets cd0 at scsibus2 targ 0 lun 0: <NECVMWar, VMware SATA CD00, 1.00> removable usb1 at uhci0: USB revision 1.0 uhub1 at usb1 configuration 1 interface 0 "VMware UHCI root hub" rev 1.00/1.00 addr 1 ppb2 at pci0 dev 21 function 0 "VMware PCIE" rev 0x01 pci3 at ppb2 bus 3 em0 at pci3 dev 0 function 0 "Intel 82574L" rev 0x00: apic 1 int 18, address 00:0c:29:20:df:4f ppb3 at pci0 dev 21 function 1 "VMware PCIE" rev 0x01 pci4 at ppb3 bus 4 ppb4 at pci0 dev 21 function 2 "VMware PCIE" rev 0x01 pci5 at ppb4 bus 5 ppb5 at pci0 dev 21 function 3 "VMware PCIE" rev 0x01 pci6 at ppb5 bus 6 ppb6 at pci0 dev 21 function 4 "VMware PCIE" rev 0x01 pci7 at ppb6 bus 7 ppb7 at pci0 dev 21 function 5 "VMware PCIE" rev 0x01 pci8 at ppb7 bus 8 ppb8 at pci0 dev 21 function 6 "VMware PCIE" rev 0x01 pci9 at ppb8 bus 9 ppb9 at pci0 dev 21 function 7 "VMware PCIE" rev 0x01 pci10 at ppb9 bus 10 ppb10 at pci0 dev 22 function 0 "VMware PCIE" rev 0x01 pci11 at ppb10 bus 11 ppb11 at pci0 dev 22 function 1 "VMware PCIE" rev 0x01 pci12 at ppb11 bus 12 ppb12 at pci0 dev 22 function 2 "VMware PCIE" rev 0x01 pci13 at ppb12 bus 13 ppb13 at pci0 dev 22 function 3 "VMware PCIE" rev 0x01 pci14 at ppb13 bus 14 ppb14 at pci0 dev 22 function 4 "VMware PCIE" rev 0x01 pci15 at ppb14 bus 15 ppb15 at pci0 dev 22 function 5 "VMware PCIE" rev 0x01 pci16 at ppb15 bus 16 ppb16 at pci0 dev 22 function 6 "VMware PCIE" rev 0x01 pci17 at ppb16 bus 17 ppb17 at pci0 dev 22 function 7 "VMware PCIE" rev 0x01 pci18 at ppb17 bus 18 ppb18 at pci0 dev 23 function 0 "VMware PCIE" rev 0x01 pci19 at ppb18 bus 19 ppb19 at pci0 dev 23 function 1 "VMware PCIE" rev 0x01 pci20 at ppb19 bus 20 ppb20 at pci0 dev 23 function 2 "VMware PCIE" rev 0x01 pci21 at ppb20 bus 21 ppb21 at pci0 dev 23 function 3 "VMware PCIE" rev 0x01 pci22 at ppb21 bus 22 ppb22 at pci0 dev 23 function 4 "VMware PCIE" rev 0x01 pci23 at ppb22 bus 23 ppb23 at pci0 dev 23 function 5 "VMware PCIE" rev 0x01 pci24 at ppb23 bus 24 ppb24 at pci0 dev 23 function 6 "VMware PCIE" rev 0x01 pci25 at ppb24 bus 25 ppb25 at pci0 dev 23 function 7 "VMware PCIE" rev 0x01 pci26 at ppb25 bus 26 ppb26 at pci0 dev 24 function 0 "VMware PCIE" rev 0x01 pci27 at ppb26 bus 27 ppb27 at pci0 dev 24 function 1 "VMware PCIE" rev 0x01 pci28 at ppb27 bus 28 ppb28 at pci0 dev 24 function 2 "VMware PCIE" rev 0x01 pci29 at ppb28 bus 29 ppb29 at pci0 dev 24 function 3 "VMware PCIE" rev 0x01 pci30 at ppb29 bus 30 ppb30 at pci0 dev 24 function 4 "VMware PCIE" rev 0x01 pci31 at ppb30 bus 31 ppb31 at pci0 dev 24 function 5 "VMware PCIE" rev 0x01 pci32 at ppb31 bus 32 ppb32 at pci0 dev 24 function 6 "VMware PCIE" rev 0x01 pci33 at ppb32 bus 33 ppb33 at pci0 dev 24 function 7 "VMware PCIE" rev 0x01 pci34 at ppb33 bus 34 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) wsmouse0 at pms0 mux 0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 uhidev0 at uhub1 port 1 configuration 1 interface 0 "VMware VMware Virtual USB Mouse" rev 1.10/1.02 addr 2 uhidev0: iclass 3/0 ums0 at uhidev0: 16 buttons, Z and W dir wsmouse1 at ums0 mux 0 uhidev1 at uhub1 port 1 configuration 1 interface 1 "VMware VMware Virtual USB Mouse" rev 1.10/1.02 addr 2 uhidev1: iclass 3/0 ums1 at uhidev1: 16 buttons, Z and W dir wsmouse2 at ums1 mux 0 uhub2 at uhub1 port 2 configuration 1 interface 0 "VMware VMware Virtual USB Hub" rev 1.10/1.00 addr 3 vscsi0 at root scsibus3 at vscsi0: 256 targets softraid0 at root scsibus4 at softraid0: 256 targets root on sd0a (996402083dafeb5f.a) swap on sd0b dump on sd0b
