Hi,
In our company, we have been using HP DL380 Gen8 as firewalls for about four years (dmesg below). We started with OpenBSD 7.0, and currently, 7.7 is in use. Since at least version 7.5, we have been experiencing random and now very frequent (almost daily) system crashes due to a kernel crash. We have tried the following to resolve this: - disabling services (BGP, OSPF, etc.) - replacing hardware (only network cards, the entire servers) - configuration changes to PF None of these helped. We also don’t have the ability to type anything in the ddb console because the entire system is frozen. Only the power button helps. The following messages have been observed so far: kernel: protection fault trap, code=0 Stopped at arptimer+0x45: movq 0x10(%r15),%rdi pmap_unwire: wiring for pmap 0xfffffd81d11cf880 va 0xc00275e000 didn't change! pmap_unwire: wiring for pmap 0xfffffd81d11cf880 va 0xc00014e000 didn't change! The most common reason for the crash is: kernel: protection fault trap, code=0 Stopped at rtable_lookup+0x29: movzbl 0x1(%rsi),%eax We are randomly running into the same situation. What information do you need? Thank you very much. Regards, Danny DMESG: OpenBSD 7.7 (GENERIC.MP) #0: Sun May 4 11:23:50 MDT 2025 r...@syspatch-77-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENER IC.MP real mem = 17127092224 (16333MB) avail mem = 16581419008 (15813MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xbfbdb000 (190 entries) bios0: vendor HP version "P70" date 09/08/2013 bios0: HP ProLiant DL380p Gen8 acpi0 at bios0: ACPI 3.0 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP SPCR MCFG HPET FFFF SPMI ERST APIC SRAT FFFF BERT HEST DMAR FFFF PCCT SSDT SSDT SSDT SSDT SSDT SSDT SSDT acpi0: wakeup devices PCI0(S5) IPT1(S5) IPT2(S5) IPT3(S5) IPT4(S5) IPT5(S5) IPT6(S5) IPT7(S5) IPT8(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimcfg0 at acpi0 acpimcfg0: addr 0xc0000000, bus 0-255 acpihpet0 at acpi0: 14318179 Hz acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.49 MHz, 06-2d-07, patch 0000071a cpu0: cpuid 1 edx=bfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CM OV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> ecx=17bee3ff<SSE3,PCLMUL,DTES64,MWAIT,DS- CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,P OPCNT,DEADLINE,AES,XSAVE,AVX> cpu0: cpuid 6 eax=77<SENSOR,ARAT> ecx=9<EFFFREQ> cpu0: cpuid 7.0 edx=9c000400<MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD> cpu0: cpuid a vers=3, gp=4, gpwidth=48, ff=3, ffwidth=48 cpu0: cpuid d.1 eax=1<XSAVEOPT> cpu0: cpuid 80000001 edx=2c100800<NXE,PAGE1GB,RDTSCP,LONG> ecx=1<LAHF> cpu0: cpuid 80000007 edx=100<ITSC> cpu0: MELTDOWN cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line 8-way L2 cache, 15MB 64b/line 20-way L3 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 99MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.52 MHz, 06-2d-07, patch 0000071a cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.51 MHz, 06-2d-07, patch 0000071a cpu2: smt 0, core 2, package 0 cpu3 at mainbus0: apid 6 (application processor) cpu3: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.57 MHz, 06-2d-07, patch 0000071a cpu3: smt 0, core 3, package 0 cpu4 at mainbus0: apid 8 (application processor) cpu4: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.54 MHz, 06-2d-07, patch 0000071a cpu4: smt 0, core 4, package 0 cpu5 at mainbus0: apid 10 (application processor) cpu5: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.65 MHz, 06-2d-07, patch 0000071a cpu5: smt 0, core 5, package 0 cpu6 at mainbus0: apid 1 (application processor) cpu6: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.53 MHz, 06-2d-07, patch 0000071a cpu6: smt 1, core 0, package 0 cpu7 at mainbus0: apid 3 (application processor) cpu7: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.68 MHz, 06-2d-07, patch 0000071a cpu7: smt 1, core 1, package 0 cpu8 at mainbus0: apid 5 (application processor) cpu8: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.76 MHz, 06-2d-07, patch 0000071a cpu8: smt 1, core 2, package 0 cpu9 at mainbus0: apid 7 (application processor) cpu9: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.83 MHz, 06-2d-07, patch 0000071a cpu9: smt 1, core 3, package 0 cpu10 at mainbus0: apid 9 (application processor) cpu10: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.72 MHz, 06-2d-07, patch 0000071a cpu10: smt 1, core 4, package 0 cpu11 at mainbus0: apid 11 (application processor) cpu11: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, 2294.63 MHz, 06-2d-07, patch 0000071a cpu11: smt 1, core 5, package 0 ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins ioapic1 at mainbus0: apid 0 pa 0xfec10000, version 20, 24 pins acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 10 (IPT1) acpiprt2 at acpi0: bus -1 (IPT2) acpiprt3 at acpi0: bus -1 (IPT3) acpiprt4 at acpi0: bus -1 (IPT4) acpiprt5 at acpi0: bus -1 (IPT5) acpiprt6 at acpi0: bus -1 (IPT6) acpiprt7 at acpi0: bus -1 (IPT7) acpiprt8 at acpi0: bus 1 (IPT8) acpiprt9 at acpi0: bus 26 (VRP1) acpiprt10 at acpi0: bus 7 (PT1A) acpiprt11 at acpi0: bus 19 (PT1B) acpiprt12 at acpi0: bus 3 (PT2A) acpiprt13 at acpi0: bus 20 (PT2B) acpiprt14 at acpi0: bus 2 (PT2C) acpiprt15 at acpi0: bus 21 (PT2D) acpiprt16 at acpi0: bus 4 (PT3A) acpiprt17 at acpi0: bus 22 (PT3B) acpiprt18 at acpi0: bus 23 (PT3C) acpiprt19 at acpi0: bus 24 (PT3D) acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000 "IPI0001" at acpi0 not configured "PNP0A06" at acpi0 not configured com0 at acpi0 COMA addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo acpicmos0 at acpi0 "PNP0303" at acpi0 not configured "PNP0F13" at acpi0 not configured "ACPI000D" at acpi0 not configured acpicpu0 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu1 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu2 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu3 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu4 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu5 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu6 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu7 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu8 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu9 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu10 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpicpu11 at acpi0: C2(350@41 mwait.3@0x20), C1(1000@1 mwait.1) acpitz0 at acpi0 acpitz0: critical temperature is 31 degC ipmi at mainbus0 not configured cpu0: using VERW MDS workaround (except on vmm entry) pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel E5 Host" rev 0x07 ppb0 at pci0 dev 1 function 0 "Intel E5 PCIE" rev 0x07 pci1 at ppb0 bus 7 ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01, msix, 6 queues, address 38:ea:a7:8f:ee:a4 ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01, msix, 6 queues, address 38:ea:a7:8f:ee:a5 ppb1 at pci0 dev 1 function 1 "Intel E5 PCIE" rev 0x07 pci2 at ppb1 bus 19 ppb2 at pci0 dev 2 function 0 "Intel E5 PCIE" rev 0x07 pci3 at ppb2 bus 3 bge0 at pci3 dev 0 function 0 "Broadcom BCM5719" rev 0x01, BCM5719 A1 (0x5719001), APE firmware NCSI 1.1.15.0: msi, address d8:9d:67:19:bc:b4 brgphy0 at bge0 phy 1: BCM5719C, rev. 0 bge1 at pci3 dev 0 function 1 "Broadcom BCM5719" rev 0x01, BCM5719 A1 (0x5719001), APE firmware NCSI 1.1.15.0: msi, address d8:9d:67:19:bc:b5 brgphy1 at bge1 phy 2: BCM5719C, rev. 0 bge2 at pci3 dev 0 function 2 "Broadcom BCM5719" rev 0x01, BCM5719 A1 (0x5719001), APE firmware NCSI 1.1.15.0: msi, address d8:9d:67:19:bc:b6 brgphy2 at bge2 phy 3: BCM5719C, rev. 0 bge3 at pci3 dev 0 function 3 "Broadcom BCM5719" rev 0x01, BCM5719 A1 (0x5719001), APE firmware NCSI 1.1.15.0: msi, address d8:9d:67:19:bc:b7 brgphy3 at bge3 phy 4: BCM5719C, rev. 0 ppb3 at pci0 dev 2 function 1 "Intel E5 PCIE" rev 0x07 pci4 at ppb3 bus 20 ppb4 at pci0 dev 2 function 2 "Intel E5 PCIE" rev 0x07 pci5 at ppb4 bus 2 ciss0 at pci5 dev 0 function 0 "Hewlett-Packard Smart Array" rev 0x01: apic 0 int 10 ciss0: 1 LD, HW rev 1, FW 4.68/4.68, 64bit fifo rro scsibus1 at ciss0: 1 targets sd0 at scsibus1 targ 0 lun 0: <HP, LOGICAL VOLUME, 4.68> sd0: 286070MB, 512 bytes/sector, 585871964 sectors ppb5 at pci0 dev 2 function 3 "Intel E5 PCIE" rev 0x07 pci6 at ppb5 bus 21 ppb6 at pci0 dev 3 function 0 "Intel E5 PCIE" rev 0x07 pci7 at ppb6 bus 4 ppb7 at pci0 dev 3 function 1 "Intel E5 PCIE" rev 0x07 pci8 at ppb7 bus 22 ppb8 at pci0 dev 3 function 2 "Intel E5 PCIE" rev 0x07 pci9 at ppb8 bus 23 ppb9 at pci0 dev 3 function 3 "Intel E5 PCIE" rev 0x07 pci10 at ppb9 bus 24 "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 0 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 1 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 2 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 3 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 4 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 5 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 6 not configured "Intel E5 DMA" rev 0x07 at pci0 dev 4 function 7 not configured "Intel E5 Address Map" rev 0x07 at pci0 dev 5 function 0 not configured "Intel E5 Error Reporting" rev 0x07 at pci0 dev 5 function 2 not configured "Intel E5 I/O APIC" rev 0x07 at pci0 dev 5 function 4 not configured ppb10 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x05 pci11 at ppb10 bus 26 ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x05: apic 8 int 21 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb11 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb5 pci12 at ppb11 bus 10 ciss1 at pci12 dev 0 function 0 "Hewlett-Packard Smart Array" rev 0x01: apic 8 int 16 ciss1: 0 LDs, HW rev 1, FW 4.68/4.68, 64bit fifo rro scsibus2 at ciss1: 8 targets ppb12 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb5 pci13 at ppb12 bus 1 "Hewlett-Packard iLO3 Slave" rev 0x05 at pci13 dev 0 function 0 not configured vga1 at pci13 dev 0 function 1 "Matrox MGA G200eH" rev 0x00 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) "Hewlett-Packard iLO3 Management" rev 0x05 at pci13 dev 0 function 2 not configured uhci0 at pci13 dev 0 function 4 "Hewlett-Packard USB" rev 0x02: apic 8 int 16 usb1 at uhci0: USB revision 1.0 uhub1 at usb1 configuration 1 interface 0 "Hewlett-Packard UHCI root hub" rev 1.00/1.00 addr 1 ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x05: apic 8 int 20 usb2 at ehci1: USB revision 2.0 uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb13 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5 pci14 at ppb13 bus 25 pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x05 pciide0 at pci0 dev 31 function 2 "Intel C600 SATA" rev 0x05: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide0: using apic 8 int 17 for native-PCI interrupt isa0 at pcib0 isadma0 at isa0 com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo com1: probed fifo depth: 0 bytes pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 vmm0 at mainbus0: VMX/EPT uhub3 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2 uhidev0 at uhub3 port 3 configuration 1 interface 0 "Lanbe KT-150QH" rev 2.00/0.00 addr 3 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 variable keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub3 port 3 configuration 1 interface 1 "Lanbe KT-150QH" rev 2.00/0.00 addr 3 uhidev1: iclass 3/1 ums0 at uhidev1: 3 buttons, Z dir wsmouse0 at ums0 mux 0 uhub4 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2 uhub5 at uhub4 port 3 configuration 1 interface 0 "Standard Microsystems product 0x2660" rev 2.00/8.01 addr 3 vscsi0 at root scsibus3 at vscsi0: 256 targets softraid0 at root scsibus4 at softraid0: 256 targets root on sd0a (2477fd4caa82bcb0.a) swap on sd0b dump on sd0b WARNING: / was not properly unmounted arp: attempt to overwrite entry for 10.18.64.147 on vlan900 by 66:51:d4:fd:06:37 on vlan400
