On Fri, Mar 01, 2024 at 12:00:29AM +0100, Alexander Bluhm wrote:
> Hi,
>
> When doing flood ping transmit from a machine and simultaneously
> ifconfig down/up in a loop, dwqe(4) interface driver crashes.
>
> dwqe_down() contains an interrupt barrier, but somehow it does not
> work. Immediately after Xspllower() a transmit interrupt is
> processed.
>
> bluhm
Unfortunately I can't see it in the dmesg, but I wonder: Is it MSIs?
Maybe the edge-triggered interrupt stays in the controller because it
isn't cleared. But things you could try are:
* Clear the IRQ status in addition to disabling them. This might not
do something in case the MSI is already in the IRQ, there are no
takebacks. But then maybe when the interrupt fires, the code path
sees the cleared status and doesn't run the tx/rx proc.
* Don't run TX/RX proc in case the interface is down?
Cheers,
Patrick
> kernel: protection fault trap, code=0
> Stopped at m_tag_delete_chain+0x30: movq 0(%rsi),%rax
>
> ddb{0}> trace
> m_tag_delete_chain(fffffd806bfa5300) at m_tag_delete_chain+0x30
> m_free(fffffd806bfa5300) at m_free+0x9e
> m_freem(fffffd806bfa5300) at m_freem+0x38
> dwqe_tx_proc(ffff800000304800) at dwqe_tx_proc+0x194
> dwqe_intr(ffff800000304800) at dwqe_intr+0x9b
> intr_handler(ffff80003f86e760,ffff8000005f4f80) at intr_handler+0x72
> Xintr_ioapic_edge36_untramp() at Xintr_ioapic_edge36_untramp+0x18f
> Xspllower() at Xspllower+0x1d
> dwqe_ioctl(ffff800000304870,80206910,ffff80003f86e990) at dwqe_ioctl+0x18c
> ifioctl(fffffd81ffabe1e8,80206910,ffff80003f86e990,ffff80003f94e550) at
> ifioctl+0x726
> sys_ioctl(ffff80003f94e550,ffff80003f86eb50,ffff80003f86eac0) at
> sys_ioctl+0x2af
> syscall(ffff80003f86eb50) at syscall+0x55b
> Xsyscall() at Xsyscall+0x128
> end of kernel
> end trace frame: 0x73ef48509270, count: -13
>
> ddb{0}> show register
> rdi 0xfffffd806bfa5300
> rsi 0xdeafbeaddeafbead
> rbp 0xffff80003f86e5f0
> rbx 0xf40
> rdx 0
> rcx 0
> rax 0xab56 __ALIGN_SIZE+0x9b56
> r8 0x90
> r9 0x24634ac __kernel_rodata_phys+0x3624ac
> r10 0xe676ed611cc13e4f
> r11 0xd2619954b795f246
> r12 0xffff800001110f48
> r13 0xfffffd8072820000
> r14 0xfffffd806bfa5300
> r15 0xfffffd805f6def00
> rip 0xffffffff81daae80 m_tag_delete_chain+0x30
> cs 0x8
> rflags 0x10282 __ALIGN_SIZE+0xf282
> rsp 0xffff80003f86e5d0
> ss 0x10
> m_tag_delete_chain+0x30: movq 0(%rsi),%rax
>
> ddb{0}> x/s version
> version: OpenBSD 7.5 (GENERIC.MP) #2: Thu Feb 29 23:42:26 CET 2024\012
> r...@ot50.obsd-lab.genua.de:/usr/src/sys/arch/amd64/compile/GENERIC.MP\012
>
> ddb{0}> ps
> PID TID PPID UID S FLAGS WAIT COMMAND
> *70039 16536 80360 0 7 0x8000003 ifconfig
> 41531 214934 36719 51 3 0x8100033 netlock ping
>
> OpenBSD 7.5 (GENERIC.MP) #2: Thu Feb 29 23:42:26 CET 2024
> r...@ot50.obsd-lab.genua.de:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 8038207488 (7665MB)
> avail mem = 7773556736 (7413MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 3.3 @ 0x769c7000 (85 entries)
> bios0: vendor American Megatrends Inc. version "1.02.10" date 06/27/2022
> efi0 at bios0: UEFI 2.7
> efi0: American Megatrends rev 0x50013
> acpi0 at bios0: ACPI 6.2
> acpi0: sleep states S0 S5
> acpi0: tablesfg0: addr 0xc0000000, bus 0-255
> acpihpet0 at acpi0: 19200000 Hz
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01, patch
> 00000017
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line
> 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 38MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.0.2.2.1.1.1, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01, patch
> 00000017
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line
> 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01, patch
> 00000017
> cpu2:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu2: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line
> 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01, patch
> 00000017
> cpu3:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu3: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line
> 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> cpu3: smt 0, core 3, package 0
> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 120 pins
> acpiprt0 at acpi0: bus 0 (PC00)
> acpiprt1 at acpi0: bus -1 (PEG0)
> acpiprt2 at acpi0: bus -1 (PEG1)
> acpiprt3 at acpi0: bus -1 (PEG2)
> acpiprt4 at acpi0: bus -1 (PEG3)
> acpiprt5 at acpi0: bus 1 (RP01)
> acpiprt6 at acpi0: bus -1 (RP02)
> acpiprt7 at acpi0: bus 2 (RP03)
> acpiprt8 at acpi0: bus 6 (RP04)
> acpiprt9 at acpi0: bus -1 (RP05)
> acpiprt10 at acpi0: bus -1 (RP06)
> acpiprt11 at acpi0: bus 7 (RP07)
> acpipci0 at acpi0 PC00: 0x00000010 0x00000011 0x00000000
> com0 at acpi0 UAR1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
> com0: console
> com1 at acpi0 UAR2 addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
> "INTC1035" at acpi0 not configured
> acpiac0 at acpi0: AC unit online
> "ACPI000E" at acpi0 not configured
> "INTC1020" at acpi0 not configured
> "INTC1020" at acpi0 not configured
> "INTC1020" at acpi0 not configured
> "INTC1020" at acpi0 not configured
> "INTC1020" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> acpibtn0 at acpi0: SLPB
> acpibtn1 at acpi0: PWRB(wakeup)
> "INTC1033" at acpi0 not configured
> "INTC1033" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> "INT33A1" at acpi0 not configured
> acpihid0 at acpi0: HIDDacpihid0: failed getting button array capability
> , 5 button array
> "MSFT8000" at acpi0 not configured
> tpm0 at acpi0 TPM_ 2.0 (TIS) addr 0xfed40000/0x5000, device 0x001b15d1 rev
> 0x16
> "ADS0345" at acpi0 not configured
> "IFX3100" at acpi0 not configured
> "TXNW1010" at acpi0 not configured
> acpipwrres0 at acpi0: WRST
> acpicpu0 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> C1(1000@1 mwait.1), PSS
> acpicpu1 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> C1(1000@1 mwait.1), PSS
> acpicpu2 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> C1(1000@1 mwait.1), PSS
> acpicpu3 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> C1(1000@1 mwait.1), PSS
> acpipwrres1 at acpi0: FN00, resource for FAN0
> acpipwrres2 at acpi0: FN01, resource for FAN1
> acpipwrres3 at acpi0: FN02, resource for FAN2
> acpipwrres4 at acpi0: FN03, resource for FAN3
> acpipwrres5 at acpi0: FN04, resource for FAN4
> acpitz0 at acpi0: critical temperature is 127 degC
> acpipwrres6 at acpi0: PIN_
> cpu0: Enhanced SpeedStep 1895 MHz: speeds: 1901, 1900, 1800, 1700, 1600,
> 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
> pci0 at mainbus0 bus 0
> 0:31:5: mem address conflict 0xfe010000/0x1000
> pchb0 at pci0 dev 0 function 0 "Intel Elkhart Lake Host" rev 0x01
> inteldrm0 at pci0 dev 2 function 0 "Intel UHD Graphics" rev 0x01
> drm0 at inteldrm0
> inteldrm0: msi, ELKHARTLAKE, gen 11
> "Intel Elkhart Lake GNA" rev 0x01 at pci0 dev 8 function 0 not configured
> xhci0 at pci0 dev 20 function 0 "Intel Elkhart Lake xHCI" rev 0x11: msi, xHCI
> 1.20
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00
> addr 1
> "Intel Elkhart Lake SRAM" rev 0x11 at pci0 dev 20 function 2 not configured
> dwiic0 at pci0 dev 21 function 0 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> int 27
> iic0 at dwiic0
> dwiic1 at pci0 dev 21 function 1 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> int 28
> iic1 at dwiic1
> dwiic2 at pci0 dev 21 function 2 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> int 29
> iic2 at dwiic2
> dwiic3 at pci0 dev 21 function 3 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> int 30
> iic3 at dwiic3
> "Intel Elkhart Lake HECI" rev 0x11 at pci0 dev 22 function 0 not configured
> ahci0 at pci0 dev 23 function 0 "Intel Elkhart Lake AHCI" rev 0x11: msi, AHCI
> 1.3.1
> ahci0: port 0: 6.0Gb/s
> ahci0: PHY offline on port 1
> scsibus1 at ahci0: 32 targets
> sd0 at scsibus1 targ 0 lun 0: <ATA, innodisk M.2(S42, S206>
> naa.524693e000367188
> sd0: 57241MB, 512 bytes/sector, 117231408 sectors, thin
> "Intel Elkhart Lake I2C" rev 0x11 at pci0 dev 24 function 0 not configured
> "Intel Elkhart Lake CAN" rev 0x11 at pci0 dev 24 function 1 not configured
> "Intel Elkhart Lake CAN" rev 0x11 at pci0 dev 24 function 2 not configured
> dwiic4 at pci0 dev 25 function 0 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> int 31
> iic4 at dwiic4
> "Intel Elkhart Lake UART" rev 0x11 at pci0 dev 25 function 2 not configured
> "Intel Elkhart Lake I2C" rev 0x11 at pci0 dev 27 function 0 not configured
> ppb0 at pci0 dev 28 function 0 "Intel Elkhart Lake PCIE" rev 0x11
> pci1 at ppb0 bus 1
> ppb1 at pci0 dev 28 function 2 "Intel Elkhart Lake PCIE" rev 0x11: msi
> pci2 at ppb1 bus 2
> ppb2 at pci2 dev 0 function 0 vendor "Pericom", unknown product 0x2303 rev
> 0x05
> pci3 at ppb2 bus 3
> ppb3 at pci3 dev 1 function 0 vendor "Pericom", unknown product 0x2303 rev
> 0x05: msi
> pci4 at ppb3 bus 4
> em0 at pci4 dev 0 function 0 "Intel I210" rev 0x03: msi, address
> f8:02:78:21:a4:35
> ppb4 at pci3 dev 2 function 0 vendor "Pericom", unknown product 0x2303 rev
> 0x05: msi
> pci5 at ppb4 bus 5
> em1 at pci5 dev 0 function 0 "Intel I210" rev 0x03: msi, address
> f8:02:78:21:a4:36
> ppb5 at pci0 dev 28 function 3 "Intel Elkhart Lake PCIE" rev 0x11: msi
> pci6 at ppb5 bus 6
> iwx0 at pci6 dev 0 function 0 "Intel Wi-Fi 6 AX210" rev 0x1a, msix
> ppb6 at pci0 dev 28 function 6 "Intel Elkhart Lake PCIE" rev 0x11: msi
> pci7 at ppb6 bus 7
> igc0 at pci7 dev 0 function 0 "Intel I225-IT" rev 0x03, msix, 4 queues,
> address 34:73:5a:ff:a5:b1
> "Intel Elkhart Lake LH2OSE" rev 0x11 at pci0 dev 29 function 0 not configured
> dwqe0 at pci0 dev 29 function 1 "Intel Elkhart Lake Ethernet" rev 0x11: rev
> 0x52, address 34:73:5a:ff:a5:b2
> eephy0 at dwqe0 phy 1: 88E1512 10/100/1000 PHY, rev. 1
> "Intel Elkhart Lake Ethernet" rev 0x11 at pci0 dev 29 function 2 not
> configured
> "Intel Elkhart Lake UART" rev 0x11 at pci0 dev 30 function 0 not configured
> pcib0 at pci0 dev 31 function 0 "Intel Elkhart Lake eSPI" rev 0x11
> azalia0 at pci0 dev 31 function 3 "Intel Elkhart Lake HD Audio" rev 0x11: msi
> azalia0: codecs: IDT/0x76e0, Intel/0x281b, using IDT/0x76e0
> audio0 at azalia0
> ichiic0 at pci0 dev 31 function 4 "Intel Elkhart Lake SMBus" rev 0x11: apic 2
> int 16
> iic5 at ichiic0
> "Intel Elkhart Lake SPI" rev 0x11 at pci0 dev 31 function 5 not configured
> isa0 at pcib0
> isadma0 at isa0
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> vmm0 at mainbus0: VMX/EPT
> efifb at mainbus0 not configured
> uhub1 at uhub0 port 3 configuration 1 interface 0 "Texas Instruments product
> 0x8027" rev 2.10/1.10 addr 2
> ugen0 at uhub0 port 5 "Intel Bluetooth" rev 2.01/0.00 addr 3
> uhub2 at uhub0 port 15 configuration 1 interface 0 "Texas Instruments product
> 0x8025" rev 3.00/1.10 addr 4
> vscsi0 at root
> scsibus2 at vscsi0: 256 targets
> softraid0 at root
> scsibus3 at softraid0: 256 targets
> root on sd0a (d6d91d6e9bdd6dbb.a) swap on sd0b dump on sd0b
> inteldrm0: 1024x768, 32bpp
> wsdisplay0 at inteldrm0 mux 1
> wsdisplay0: screen 0-5 added (std, vt100 emulation)
> iwx0: hw rev 0x420, fw 77.f92b5fed.0, pnvm 181407b3, address 8c:b8:7e:ee:89:0f
> <4>arp: attempt to add entry for 10.0.1.1 on igc0 by 00:30:48:57:ed:0a on
> dwqe0
>
