https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255104
Bug ID: 255104 Summary: FreeBSD 13.0-RELEASE panic/crash with ipfw/dummynet/divert & wlan Product: Base System Version: 13.0-STABLE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: b...@freebsd.org Reporter: ku...@gentoo.org I have upgraded my router appliance to FreeBSD 13.0-RELEASE and when using IPFW + dummynet(4) + divert(4), I can trigger the kernel to panic in a very random fashion. Background on my setup: - Hardware is a Protectli FW6C (https://protectli.com/product/fw6c/) * 16GB RAM * KINGSTON SUV500MS120G on /dev/ada0 * 6x Intel 82583V GbE network ports supported by em(4) [em0 to em5] * Custom-added Qualcom AR9462 on ath0/wlan0 - Custom kernel config installed in /boot/kernel.custom * Also a /boot/CUSTOM symlink pointing to /boot/kernel.custom - em0 is WAN, DHCP via dhclient(8) to my cable modem - em1 is LAN, connected to a Netgear switch - wlan0 is wireless LAN on a separate RFC1918 subnet from em1 - Firewall setup is IPFW-based * Uses in-kernel NAT for em1 and wlan0 subnets * Uses dummynet(4) for fq_codel shaping * Uses divert(4) socket to route packets to Snort for inline inspection Synopsis of what causes the crash: - Having Snort up and running in a tmux session - wlan0 is active and has a client station connected - ipfw divert(4) socket is active, feeding packets to Snort - Sending/receiving WLAN traffic will eventually cause a random panic/reboot - Traffic on the LAN on em1 does NOT appear to trigger a crash (note, see crash #4) Here are samples of the crashes. I do not have the original kernel for some of these, so I cannot generate full backtraces, but I do have several of the core dumps under /var/crash. Let me know what is needed to help debug this. Note, I feel that the issue highlighted in PR#255069 may be related somehow. I also tried patch D29772 posted in PR#255041, and that had no effect. Crash #6 is using this patched kernel, so I can run kgdb against it if needed. Crash #1 (Only kgdb backtrace is available): #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=<optimized out>) at ../../../kern/kern_shutdown.c:399 #2 0xffffffff8074e645 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff8074eac0 in vpanic (fmt=<optimized out>, ap=<optimized out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff8074e8c3 in panic (fmt=<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80ad2037 in trap_fatal (frame=0xfffffe00dc46d8e0, eva=8) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80ad2089 in trap_pfault (frame=frame@entry=0xfffffe00dc46d8e0, usermode=false, signo=<optimized out>, signo@entry=0x0, ucode=<optimized out>, ucode@entry=0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80ad1709 in trap (frame=0xfffffe00dc46d8e0) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 0xffffffff814f00a5 in dummynet_task () from /boot/CUSTOM/dummynet.ko #10 0xffffffff807aeda1 in taskqueue_run_locked (queue=0x8962c, queue@entry=0xfffff8000b02d300) at ../../../kern/subr_taskqueue.c:476 #11 0xffffffff807b00bc in taskqueue_thread_loop (arg=<optimized out>, arg@entry=0xffffffff814fa048 <dn_tq>) at ../../../kern/subr_taskqueue.c:793 #12 0xffffffff8070e05d in fork_exit (callout=0xffffffff807b0010 <taskqueue_thread_loop>, arg=0xffffffff814fa048 <dn_tq>, frame=0xfffffe00dc46db00) at ../../../kern/kern_fork.c:1069 #13 <signal handler called> Crash #2 (kgdb backtrace data unavailable): Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x8 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff814f00a5 stack pointer = 0x28:0xfffffe00dc46d9a0 frame pointer = 0x28:0xfffffe00dc46da00 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (dummynet) trap number = 12 panic: page fault cpuid = 0 time = 1618402444 KDB: stack backtrace: #0 0xffffffff8079b0b5 at kdb_backtrace+0x65 #1 0xffffffff8074ea51 at vpanic+0x181 #2 0xffffffff8074e8c3 at panic+0x43 #3 0xffffffff80ad2037 at trap_fatal+0x387 #4 0xffffffff80ad2089 at trap_pfault+0x49 #5 0xffffffff80ad1709 at trap+0x259 #6 0xffffffff80aaa4e8 at calltrap+0x8 #7 0xffffffff807aeda1 at taskqueue_run_locked+0x181 #8 0xffffffff807b00bc at taskqueue_thread_loop+0xac #9 0xffffffff8070e05d at fork_exit+0x7d #10 0xffffffff80aab4ee at fork_trampoline+0xe Uptime: 9m23s Dumping 787 out of 16144 MB: (CTRL-C to abort) ..3%..11%..21%..31%..41%..51%..61%..72%..82%..92% Crash #3 (this happened when sending Ctrl+C to the Snort process): Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x8 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff807ec20c stack pointer = 0x28:0xfffffe011d7d07d0 frame pointer = 0x28:0xfffffe011d7d0810 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 86334 (snort) trap number = 12 panic: page fault cpuid = 0 time = 1618439898 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05a37 at trap_fatal+0x387 #4 0xffffffff80b05a89 at trap_pfault+0x49 #5 0xffffffff80b05109 at trap+0x259 #6 0xffffffff80addee8 at calltrap+0x8 #7 0xffffffff807eaf68 at sbdestroy+0x18 #8 0xffffffff807edd39 at sofree+0x309 #9 0xffffffff807ee824 at soclose+0x2e4 #10 0xffffffff806f8a91 at _fdrop+0x11 #11 0xffffffff806fbdcb at closef+0x24b #12 0xffffffff806f8d92 at closefp+0x82 #13 0xffffffff80b0621c at amd64_syscall+0x10c #14 0xffffffff80ade80e at fast_syscall_common+0xf8 Uptime: 21m57s Dumping 786 out of 16146 MB:..3%..11%..21%..31%..41%..51%..62%..72%..82%..92% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=<optimized out>) at ../../../kern/kern_shutdown.c:399 #2 0xffffffff80751e85 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=<optimized out>, ap=<optimized out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05a37 in trap_fatal (frame=0xfffffe011d7d0710, eva=8) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b05a89 in trap_pfault (frame=frame@entry=0xfffffe011d7d0710, usermode=false, signo=<optimized out>, signo@entry=0x0, ucode=<optimized out>, ucode@entry=0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80b05109 in trap (frame=0xfffffe011d7d0710) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 sbcut_internal (sb=sb@entry=0xfffff802fa2d68a8, len=3404) at ../../../kern/uipc_sockbuf.c:1491 #10 0xffffffff807eaf68 in sbflush_internal (sb=0xfffff802fa2d68a8, sb@entry=0xfffff802fa2d6760) at ../../../kern/uipc_sockbuf.c:1431 #11 sbrelease_internal (sb=0xfffff802fa2d68a8, sb@entry=0xfffff802fa2d6760, so=0xfffff802fa2d6760, so@entry=0xfffff802fa2d68a8) at ../../../kern/uipc_sockbuf.c:721 #12 sbdestroy (sb=sb@entry=0xfffff802fa2d68a8, so=so@entry=0xfffff802fa2d6760) at ../../../kern/uipc_sockbuf.c:749 #13 0xffffffff807edd39 in sofree (so=so@entry=0xfffff802fa2d6760) at ../../../kern/uipc_socket.c:1158 #14 0xffffffff807ee824 in soclose (so=0xfffff802fa2d6760) at ../../../kern/uipc_socket.c:1235 #15 0xffffffff806f8a91 in fo_close (fp=fp@entry=0xfffff80010895500, td=0xd4c, td@entry=0xfffffe012053a000) at ../../../sys/file.h:377 #16 _fdrop (fp=fp@entry=0xfffff80010895500, td=0xd4c, td@entry=0xfffffe012053a000) at ../../../kern/kern_descrip.c:3510 #17 0xffffffff806fbdcb in closef (fp=fp@entry=0xfffff80010895500, td=td@entry=0xfffffe012053a000) at ../../../kern/kern_descrip.c:2828 #18 0xffffffff806f8d92 in closefp_impl (fdp=<optimized out>, fd=4, fp=0xfffff80010895500, td=0xfffffe012053a000, audit=true) at ../../../kern/kern_descrip.c:1271 #19 closefp (fdp=<optimized out>, fd=4, fp=0xfffff80010895500, td=0xfffffe012053a000, holdleaders=<optimized out>, audit=true) at ../../../kern/kern_descrip.c:1328 #20 0xffffffff80b0621c in syscallenter (td=0xfffffe012053a000) at ../../../amd64/amd64/../../kern/subr_syscall.c:189 #21 amd64_syscall (td=0xfffffe012053a000, traced=0) at ../../../amd64/amd64/trap.c:1156 #22 <signal handler called> #23 0x000000080915b40a in ?? () Backtrace stopped: Cannot access memory at address 0x7fffff4b1458 Crash #4 (based on the stacktrace, this may have been caused by emX traffic): NOTE: I use an out-of-tree copy of em-7.7.8 from Intel upstream, modifed to compile under FreeBSD 13.0 (changes are trivial). Fatal trap 9: general protection fault while in kernel mode cpuid = 1; apic id = 02 instruction pointer = 0x20:0xffffffff8086e9dc stack pointer = 0x28:0xfffffe00c5b9f840 frame pointer = 0x28:0xfffffe00c5b9f890 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em0 que) trap number = 9 panic: general protection fault cpuid = 1 time = 1618440500 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05a37 at trap_fatal+0x387 #4 0xffffffff80b055cf at trap+0x71f #5 0xffffffff80addee8 at calltrap+0x8 #6 0xffffffff8088c488 at netisr_dispatch_src+0xc8 #7 0xffffffff8086ddd9 at ether_input+0x69 #8 0xffffffff8086a69a at if_input+0xa #9 0xffffffff81b1f000 at em_rxeof+0x260 #10 0xffffffff81b20380 at em_handle_que+0x40 #11 0xffffffff807b25e1 at taskqueue_run_locked+0x181 #12 0xffffffff807b38fc at taskqueue_thread_loop+0xac #13 0xffffffff8071189d at fork_exit+0x7d #14 0xffffffff80adeeee at fork_trampoline+0xe Uptime: 9m14s Dumping 819 out of 16146 MB:..2%..12%..22%..32%..42%..51%..61%..71%..81%..92% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=<optimized out>) at ../../../kern/kern_shutdown.c:399 #2 0xffffffff80751e85 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=<optimized out>, ap=<optimized out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05a37 in trap_fatal (frame=0xfffffe00c5b9f780, eva=0) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b055cf in trap (frame=0xfffffe00c5b9f780) at ../../../amd64/amd64/trap.c:576 #7 <signal handler called> #8 ether_input_internal (ifp=0x5f48844900310210, m=0xfffff8039a9e9d00) at ../../../net/if_ethersubr.c:524 #9 ether_nh_input (m=0xfffff8039a9e9d00) at ../../../net/if_ethersubr.c:739 #10 0xffffffff8088c488 in netisr_dispatch_src (proto=proto@entry=5, source=<optimized out>, source@entry=0, m=m@entry=0xfffff8039a9e9d00) at ../../../net/netisr.c:1143 #11 0xffffffff8088c76f in netisr_dispatch (proto=2594086144, proto@entry=5, m=0x2d, m@entry=0xfffff8039a9e9d00) at ../../../net/netisr.c:1234 #12 0xffffffff8086ddd9 in ether_input (ifp=<optimized out>, m=0xfffff8039a9e9d00) at ../../../net/if_ethersubr.c:830 #13 0xffffffff8086a69a in if_input (ifp=0xfffff8039a9e9d00, sendmp=0x0) at ../../../net/if.c:4391 #14 0xffffffff81b1f000 in em_rxeof () from /boot/modules/if_em_updated.ko #15 0xffffffff81b20380 in em_handle_que () from /boot/modules/if_em_updated.ko #16 0xffffffff807b25e1 in taskqueue_run_locked (queue=0xfffff80017500200, queue@entry=0xfffff80002bdfa00) at ../../../kern/subr_taskqueue.c:476 #17 0xffffffff807b38fc in taskqueue_thread_loop (arg=<optimized out>, arg@entry=0xfffffe002014e6a0) at ../../../kern/subr_taskqueue.c:793 #18 0xffffffff8071189d in fork_exit ( callout=0xffffffff807b3850 <taskqueue_thread_loop>, arg=0xfffffe002014e6a0, frame=0xfffffe00c5b9fb00) at ../../../kern/kern_fork.c:1069 #19 <signal handler called> Crash #5: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 02 fault virtual address = 0x0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8047ae0d stack pointer = 0x28:0xfffffe001d3fc550 frame pointer = 0x28:0xfffffe001d3fc590 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi1: netisr 1) trap number = 12 panic: page fault cpuid = 1 time = 1618441084 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05a37 at trap_fatal+0x387 #4 0xffffffff80b05a89 at trap_pfault+0x49 #5 0xffffffff80b05109 at trap+0x259 #6 0xffffffff80addee8 at calltrap+0x8 #7 0xffffffff808a73a3 at ieee80211_parent_xmitpkt+0x13 #8 0xffffffff808b988e at ieee80211_vap_pkt_send_dest+0x25e #9 0xffffffff808ba606 at ieee80211_vap_transmit+0x1d6 #10 0xffffffff8086d82b at ether_output_frame+0xab #11 0xffffffff8086d727 at ether_output+0x6b7 #12 0xffffffff808eb2e9 at ip_output_send+0x109 #13 0xffffffff808eb062 at ip_output+0x12a2 #14 0xffffffff808e8164 at ip_forward+0x394 #15 0xffffffff808e7d89 at ip_input+0x6c9 #16 0xffffffff8088cc1b at swi_net+0x12b #17 0xffffffff80714abd at ithread_loop+0x24d Uptime: 3m18s Dumping 849 out of 16146 MB:..2%..12%..21%..31%..42%..51%..61%..72%..81%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=<optimized out>) at ../../../kern/kern_shutdown.c:399 #2 0xffffffff80751e85 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=<optimized out>, ap=<optimized out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05a37 in trap_fatal (frame=0xfffffe001d3fc490, eva=0) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b05a89 in trap_pfault (frame=frame@entry=0xfffffe001d3fc490, usermode=false, signo=<optimized out>, signo@entry=0x0, ucode=<optimized out>, ucode@entry=0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80b05109 in trap (frame=0xfffffe001d3fc490) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 ath_transmit (ic=<optimized out>, m=0xfffff801ed556200) at ../../../dev/ath/if_ath.c:3516 #10 0xffffffff808a73a3 in ieee80211_parent_xmitpkt (ic=0x0, ic@entry=0xfffffe00d844f000, m=m@entry=0xfffff8001e808300) at ../../../net80211/ieee80211_freebsd.c:717 #11 0xffffffff808b988e in ieee80211_vap_pkt_send_dest ( vap=vap@entry=0xfffff8001e266000, m=m@entry=0xfffff8001e808300, ni=ni@entry=0xfffffe012c7b1000) at ../../../net80211/ieee80211_output.c:317 #12 0xffffffff808ba606 in ieee80211_start_pkt (vap=0xfffff8001e266000, m=0xfffff8001e808300) at ../../../net80211/ieee80211_output.c:474 #13 ieee80211_vap_transmit (ifp=<optimized out>, m=<optimized out>) at ../../../net80211/ieee80211_output.c:534 #14 0xffffffff8086d82b in ether_output_frame ( ifp=ifp@entry=0xfffff8001e188000, m=0xfffffe012c7b1000) at ../../../net/if_ethersubr.c:511 #15 0xffffffff8086d727 in ether_output (ifp=<optimized out>, m=0xfffffe012c7b1000, dst=0xfffffe001d3fc8e0, ro=<optimized out>) at ../../../net/if_ethersubr.c:438 #16 0xffffffff808eb2e9 in ip_output_send (inp=inp@entry=0x0, ifp=0xfffff8001e188000, m=m@entry=0xfffff8001e808300, gw=<optimized out>, gw@entry=0xfffffe001d3fc8e0, ro=<optimized out>, ro@entry=0xfffffe001d3fc8c0, stamp_tag=<optimized out>) at ../../../netinet/ip_output.c:275 #17 0xffffffff808eb062 in ip_output (m=m@entry=0xfffff8001e808300, opt=<optimized out>, opt@entry=0x0, ro=<optimized out>, ro@entry=0xfffffe001d3fc8c0, flags=flags@entry=1, imo=imo@entry=0x0, inp=<optimized out>, inp@entry=0x0) at ../../../netinet/ip_output.c:812 #18 0xffffffff808e8164 in ip_forward (m=0xfffff8001e808300, srcrt=<optimized out>) at ../../../netinet/ip_input.c:1067 #19 0xffffffff808e7d89 in ip_input (m=0x0) at ../../../netinet/ip_input.c:789 #20 0xffffffff8088cc1b in netisr_process_workstream_proto ( nwsp=<optimized out>, proto=1) at ../../../net/netisr.c:919 #21 swi_net (arg=<optimized out>) at ../../../net/netisr.c:966 #22 0xffffffff80714abd in intr_event_execute_handlers (p=<optimized out>, ie=0xfffff80002826b00) at ../../../kern/kern_intr.c:1168 #23 ithread_execute_handlers (p=<optimized out>, ie=0xfffff80002826b00) at ../../../kern/kern_intr.c:1181 #24 ithread_loop (arg=arg@entry=0xfffff80002833ac0) at ../../../kern/kern_intr.c:1269 #25 0xffffffff8071189d in fork_exit ( callout=0xffffffff80714870 <ithread_loop>, arg=0xfffff80002833ac0, frame=0xfffffe001d3fcb00) at ../../../kern/kern_fork.c:1069 #26 <signal handler called> Crash #6: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 02 fault virtual address = 0x388 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8088cc07 stack pointer = 0x28:0xfffffe001d3fc9c0 frame pointer = 0x28:0xfffffe001d3fca20 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi1: netisr 1) trap number = 12 panic: page fault cpuid = 1 time = 1618528473 KDB: stack backtrace: #0 0xffffffff8079e8f5 at kdb_backtrace+0x65 #1 0xffffffff80752291 at vpanic+0x181 #2 0xffffffff80752103 at panic+0x43 #3 0xffffffff80b05d07 at trap_fatal+0x387 #4 0xffffffff80b05d59 at trap_pfault+0x49 #5 0xffffffff80b053d9 at trap+0x259 #6 0xffffffff80ade1b8 at calltrap+0x8 #7 0xffffffff80714abd at ithread_loop+0x24d #8 0xffffffff8071189d at fork_exit+0x7d #9 0xffffffff80adf1be at fork_trampoline+0xe Uptime: 2m28s Dumping 781 out of 16146 MB:..3%..11%..21%..31%..41%..52%..62%..72%..82%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory. (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 #1 doadump (textdump=<optimized out>) at ../../../kern/kern_shutdown.c:399 #2 0xffffffff80751e85 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:486 #3 0xffffffff80752300 in vpanic (fmt=<optimized out>, ap=<optimized out>) at ../../../kern/kern_shutdown.c:919 #4 0xffffffff80752103 in panic (fmt=<unavailable>) at ../../../kern/kern_shutdown.c:843 #5 0xffffffff80b05d07 in trap_fatal (frame=0xfffffe001d3fc900, eva=904) at ../../../amd64/amd64/trap.c:915 #6 0xffffffff80b05d59 in trap_pfault (frame=frame@entry=0xfffffe001d3fc900, usermode=false, signo=<optimized out>, signo@entry=0x0, ucode=<optimized out>, ucode@entry=0x0) at ../../../amd64/amd64/trap.c:732 #7 0xffffffff80b053d9 in trap (frame=0xfffffe001d3fc900) at ../../../amd64/amd64/trap.c:398 #8 <signal handler called> #9 0xffffffff8088cc07 in netisr_process_workstream_proto ( nwsp=<optimized out>, proto=1) at ../../../net/netisr.c:918 #10 swi_net (arg=<optimized out>) at ../../../net/netisr.c:966 #11 0xffffffff80714abd in intr_event_execute_handlers (p=<optimized out>, ie=0xfffff80002826b00) at ../../../kern/kern_intr.c:1168 #12 ithread_execute_handlers (p=<optimized out>, ie=0xfffff80002826b00) at ../../../kern/kern_intr.c:1181 #13 ithread_loop (arg=arg@entry=0xfffff80002833ac0) at ../../../kern/kern_intr.c:1269 #14 0xffffffff8071189d in fork_exit ( callout=0xffffffff80714870 <ithread_loop>, arg=0xfffff80002833ac0, frame=0xfffffe001d3fcb00) at ../../../kern/kern_fork.c:1069 #15 <signal handler called> ----------------------------------------------------------------------- I suspect the underlying flaw is somehow tied to an interaction with divert(8) and dummynet(8) and the wlan0 adapter. Standard LAN traffic does not seem to trigger the panic, or at least trigger it as easily. But WLAN traffic does trigger it very easily, usually within a minute or two of turning on the divert(8) rule, connecting a wireless station, and generating some wireless traffic. I also suspect Snort is applying memory pressure somehow. I am using the standard Talos ruleset (30-day delayed release, several months old). This is how I start Snort-2.9.17: snort -c /usr/local/etc/snort/snort.conf -i em0 -k none -A console -Q --daq ipfw --daq-mode inline --daq-var port=8000 And this is the divert(8) rule: ipfw add 00049 divert 8000 all from any to any via em0 This is my NAT/dummynet configuration from the firewall: /sbin/ipfw nat 1 config if em0 deny_in same_ports unreg_only reset /sbin/ipfw pipe 1 config bw 294MBit/s burst 1048576 # Download pipe /sbin/ipfw pipe 2 config bw 12MBit/s # Upload pipe /sbin/ipfw sched 1 config pipe 1 type fq_codel target 5ms quantum 6000 flows 2048 interval 300 limit 15360 ecn /sbin/ipfw sched 2 config pipe 2 type fq_codel ecn /sbin/ipfw queue 01 config sched 2 weight 100 # Outbound TCP ACK /sbin/ipfw queue 02 config sched 1 weight 100 # Inbound TCP ACK /sbin/ipfw queue 03 config sched 2 weight 90 # Outbound HTTP/HTTPS/RSYNC /sbin/ipfw queue 04 config sched 1 weight 90 # Inbound HTTP/HTTPS/RSYNC /sbin/ipfw queue 05 config sched 2 weight 85 # Outbound DNS /sbin/ipfw queue 06 config sched 1 weight 85 # Inbound DNS /sbin/ipfw queue 07 config sched 2 weight 65 # Outbound Steam Client /sbin/ipfw queue 08 config sched 1 weight 65 # Inbound Steam Client /sbin/ipfw queue 09 config sched 2 weight 55 # Outbound IMAP/POP3/SMTP /sbin/ipfw queue 10 config sched 1 weight 55 # Inbound IMAP/POP3/SMTP That's about all I can think that is relevant. Please let me know if any additional information is needed. The system is rolled back to FreeBSD 12.2, but I am keeping the FreeBSD 13.0 boot environment, so I can easily reboot into 13.0 and try any patches out. -- You are receiving this mail because: You are the assignee for the bug. _______________________________________________ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"