Hello,

I have analyzed the trace and reproduced the deadlock mechanism in code.

The issue is an ABBA deadlock between rtnl_lock and dev->mt76.mutex:
1. mt792x_mac_work holds dev->mt76.mutex while looping on register reads during 
USB timeouts (-110).
2. USB unbind calls mt792xu_disconnect -> mt76_unregister_device, which takes 
rtnl_lock.
3. Under rtnl_lock, ieee80211_set_disassoc calls mt7921_abort_roc, which waits 
for dev->mt76.mutex.
4. Neither thread can finish, blocking rtnl_lock forever. NetworkManager and 
shutdown hang in D state.

I prepared a patch:
- In mt792xu_disconnect(): set MT76_REMOVED flag immediately and cancel pending 
workers before calling mt76_unregister_device(). Once MT76_REMOVED is set, any 
pending USB requests fail instantly with -EIO instead of timing out.
- In mt7921_abort_roc(): check MT76_REMOVED. If set, return immediately without 
waiting for the mutex.

I attached lp2167595_mt7921u_usb_disconnect_deadlock.patch.
I also built test modules for kernel 7.0.0-31-generic. If you would like to 
test them on your Dell machine, I can share the archive or test PPA.


** Description changed:

- Package: linux (Ubuntu)
- Release: Ubuntu 26.04.1 LTS (resolute)
- Kernel: Ubuntu 7.0.0-31.31-generic 7.0.14
- Hardware: Dell Precision 3650 Tower (BIOS 1.48.0)
- Wi-Fi: ALFA USB adapter, MediaTek MT7921AU, USB ID 0e8d:7961, driver mt7921u
-        (on xHCI port 1-9, high-speed; kernel cmdline has 
usbcore.autosuspend=-1)
- Firmware: linux-firmware 20260319.git217ca6e4.1ubuntu
-           WM Firmware Version ____010000, Build Time 20260224110949
+ [ Impact ]
+ When an MT7921AU USB Wi-Fi dongle (e.g. 0e8d:7961) experiences communication 
stalls (USB -110 ETIMEDOUT errors), mt792x_mac_work holds dev->mt76.mutex while 
looping on register reads. When USB core detects the stall and triggers unbind, 
mt792xu_disconnect calls mt76_unregister_device, which takes rtnl_lock and 
calls mt7921_abort_roc.
  
- [Impact]
- After the adapter stops answering USB control requests, the USB core queues a
- device reset. The reset unbinds mt7921u, and mt76_unregister_device() blocks 
in
- mt7921_abort_roc() waiting for the device mutex while holding RTNL.
- NetworkManager, ip, and every other process that needs RTNL then sits in
- D state for good. The machine cannot be powered off cleanly and needs a hard
- power-off.
+ mt7921_abort_roc then tries to acquire dev->mt76.mutex, which is already
+ held by the stuck mac_work thread. This creates an ABBA deadlock between
+ rtnl_lock and mt76.mutex.
  
- [Timeline, 2026-09-15]
- - 19:05:31  first "mt7921u 1-9:1.3: vendor request req:63 off:d02c 
failed:-110"
- - ~3 s per failed request, 84 failures in total until 19:09:55
- - 19:09:43  hung_task reports 8 blocked tasks (NetworkManager, ip, kworkers,
-             user-space apps), all "blocked on a mutex likely owned by task
-             kworker/7:1:304"
- - 19:09:55  last journal entry; hard power-off required
+ As a result:
+ - mac_work waits on USB timeouts with mt76.mutex held.
+ - mt792xu_disconnect waits for mt76.mutex under rtnl_lock.
+ - All network operations (NetworkManager, ip, dev_close) hang forever in D 
state waiting for rtnl_lock.
+ - The system cannot power off or reboot cleanly without a hard reset.
  
- [Blocker: kworker/7:1 (USB reset -> unbind -> unregister)]
- task:kworker/7:1     state:D stack:0     pid:304   tgid:304   ppid:2      
task_flags:0x4288060 flags:0x00080000
- Workqueue: events __usb_queue_reset_device
- Call Trace:
-  <TASK>
-  __schedule+0x2bb/0x650
-  schedule+0x27/0xb0
-  schedule_preempt_disabled+0x15/0x30
-  __mutex_lock.constprop.0+0x550/0xaf0
-  __mutex_lock_slowpath+0x13/0x20
-  mutex_lock+0x3b/0x50
-  mt7921_abort_roc+0x3d/0xb0 [mt7921_common]
-  mt7921_mgd_complete_tx+0x1c/0x30 [mt7921_common]
-  drv_mgd_complete_tx.part.0+0x57/0x180 [mac80211]
-  ieee80211_set_disassoc+0x7fb/0x8a0 [mac80211]
-  ieee80211_mgd_deauth+0x148/0x590 [mac80211]
-  ieee80211_deauth+0x18/0x30 [mac80211]
-  cfg80211_mlme_deauth+0xa8/0x220 [cfg80211]
-  cfg80211_mlme_down+0x60/0x90 [cfg80211]
-  cfg80211_disconnect+0x1a6/0x230 [cfg80211]
-  cfg80211_leave+0x237/0x290 [cfg80211]
-  cfg80211_netdev_notifier_call+0x185/0x4d0 [cfg80211]
-  notifier_call_chain+0x62/0xf0
-  raw_notifier_call_chain+0x16/0x30
-  call_netdevice_notifiers_info+0x50/0x80
-  __dev_close_many+0x65/0x220
-  netif_close_many+0xb7/0x1b0
-  netif_close+0x70/0xa0
-  dev_close+0x38/0xb0
-  cfg80211_shutdown_all_interfaces+0x50/0x100 [cfg80211]
-  ieee80211_remove_interfaces+0x47/0x220 [mac80211]
-  ieee80211_unregister_hw+0x4a/0x140 [mac80211]
-  mt76_unregister_device+0x63/0x80 [mt76]
-  mt792xu_disconnect+0x32/0x90 [mt792x_usb]
-  usb_unbind_interface+0x9b/0x2c0
-  device_remove+0x68/0x80
-  device_release_driver_internal+0x1fb/0x260
-  device_release_driver+0x12/0x20
-  usb_forced_unbind_intf+0x96/0xe0
-  usb_reset_device+0xf4/0x300
-  __usb_queue_reset_device+0x3b/0x60
-  process_one_work+0x1ac/0x3d0
-  worker_thread+0x1b8/0x360
-  kthread+0xf7/0x130
-  ret_from_fork+0x195/0x2a0
-  ret_from_fork_asm+0x1a/0x30
-  </TASK>
+ [ Fix ]
+ 1. In mt792xu_disconnect(), set MT76_REMOVED, MT76_RESET, and MT76_MCU_RESET 
flags and wake pending waitqueues before calling mt76_unregister_device(). 
Setting MT76_REMOVED causes all pending and subsequent USB register requests to 
fail immediately with -EIO instead of waiting for 3-second timeouts.
+ 2. In mt792xu_disconnect(), synchronously cancel all workers (mac_work, 
ps_work, wake_work, reset_work, init_work) prior to unregistration.
+ 3. In mt7921_abort_roc() (and mt7925_abort_roc()), check if MT76_REMOVED is 
set. If the device was removed, clear MT76_STATE_ROC and return 0 immediately 
without taking dev->mt76.mutex.
  
- [Victim: NetworkManager waiting for RTNL (first frames)]
- INFO: task NetworkManager:3094 blocked for more than 122 seconds.
-       Tainted: G           OE       7.0.0-31-generic #31-Ubuntu
- "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- task:NetworkManager  state:D stack:0     pid:3094  tgid:3094  ppid:1      
task_flags:0x400100 flags:0x00080000
- Call Trace:
-  <TASK>
-  __schedule+0x2bb/0x650
-  schedule+0x27/0xb0
-  schedule_preempt_disabled+0x15/0x30
-  __mutex_lock.constprop.0+0x550/0xaf0
-  __mutex_lock_slowpath+0x13/0x20
-  mutex_lock+0x3b/0x50
-  rtnl_lock+0x15/0x20
-  nl80211_prepare_wdev_dump+0x8a/0x250 [cfg80211]
-  nl80211_dump_station+0x80/0x4c0 [cfg80211]
-  genl_dumpit+0x36/0x90
-  netlink_dump+0x1a2/0x400
- INFO: task NetworkManager:3094 is blocked on a mutex likely owned by task 
kworker/7:1:304.
- task:kworker/7:1     state:D stack:0     pid:304   tgid:304   ppid:2      
task_flags:0x4288060 flags:0x00080000
- Workqueue: events __usb_queue_reset_device
- Call Trace:
-  <TASK>
-  __schedule+0x2bb/0x650
-  schedule+0x27/0xb0
-  schedule_preempt_disabled+0x15/0x30
-  __mutex_lock.constprop.0+0x550/0xaf0
-  __mutex_lock_slowpath+0x13/0x20
-  mutex_lock+0x3b/0x50
-  mt7921_abort_roc+0x3d/0xb0 [mt7921_common]
-  mt7921_mgd_complete_tx+0x1c/0x30 [mt7921_common]
-  drv_mgd_complete_tx.part.0+0x57/0x180 [mac80211]
-  ieee80211_set_disassoc+0x7fb/0x8a0 [mac80211]
-  ieee80211_mgd_deauth+0x148/0x590 [mac80211]
-  ieee80211_deauth+0x18/0x30 [mac80211]
-  cfg80211_mlme_deauth+0xa8/0x220 [cfg80211]
-  cfg80211_mlme_down+0x60/0x90 [cfg80211]
-  cfg80211_disconnect+0x1a6/0x230 [cfg80211]
-  cfg80211_leave+0x237/0x290 [cfg80211]
-  cfg80211_netdev_notifier_call+0x185/0x4d0 [cfg80211]
-  notifier_call_chain+0x62/0xf0
-  raw_notifier_call_chain+0x16/0x30
-  call_netdevice_notifiers_info+0x50/0x80
-  __dev_close_many+0x65/0x220
-  netif_close_many+0xb7/0x1b0
-  netif_close+0x70/0xa0
-  dev_close+0x38/0xb0
-  cfg80211_shutdown_all_interfaces+0x50/0x100 [cfg80211]
-  ieee80211_remove_interfaces+0x47/0x220 [mac80211]
-  ieee80211_unregister_hw+0x4a/0x140 [mac80211]
-  mt76_unregister_device+0x63/0x80 [mt76]
-  mt792xu_disconnect+0x32/0x90 [mt792x_usb]
-  usb_unbind_interface+0x9b/0x2c0
-  device_remove+0x68/0x80
-  device_release_driver_internal+0x1fb/0x260
-  device_release_driver+0x12/0x20
-  usb_forced_unbind_intf+0x96/0xe0
-  usb_reset_device+0xf4/0x300
-  __usb_queue_reset_device+0x3b/0x60
-  process_one_work+0x1ac/0x3d0
-  worker_thread+0x1b8/0x360
-  kthread+0xf7/0x130
-  ret_from_fork+0x195/0x2a0
-  ret_from_fork_asm+0x1a/0x30
-  </TASK>
+ [ Test Plan ]
+ 1. Boot kernel 7.0.0-31-generic with patched mt7921u / mt792x-usb modules.
+ 2. Insert MT7921AU USB Wi-Fi adapter (0e8d:7961) and connect to an AP.
+ 3. Physically disconnect the USB adapter while network traffic is active, or 
trigger bus reset while requests are pending.
+ 4. Verify in dmesg:
+    - Device disconnect completes cleanly without call traces.
+    - rtnl_lock is not blocked; NetworkManager and ip link operate normally.
+    - System reboots and powers off cleanly without hanging in D state.
  
- [Already verified]
- - The kernel already has the fixes from 7.0.0-28 ("mt7921: fix potential
-   deadlock in mt7921_roc_abort_sync" / CVE-2026-53101, "mt76: fix deadlock in
-   remain-on-channel", "mt7921: fix ROC abort flow interruption in
-   mt7921_roc_work"). The hang still happens on 7.0.0-31, so this is a
-   different path: USB reset/disconnect while the bus is hung.
- - The upstream series "wifi: mt76: mt792x: harden USB reset and disconnect
-   paths" (Sean Wang, 2026-06-13, 6 patches, Fixes: 0d2afe09fad5 "mt76: mt7921:
-   add mt7921u driver") describes exactly these symptoms: repeated -110 vendor
-   request timeouts, teardown blocked by failing register access, and quiescing
-   the USB paths in mt792xu_disconnect. Please consider it (and its dependency
-   "wifi: mt76: mt76u: use a threaded NAPI for the RX path") for an SRU once it
-   lands upstream.
+ [ Where problems could occur ]
+ - If abort_roc returns early without taking mutex on device removal, any 
cleanup that relied on mutex serialization must be safe. Since MT76_REMOVED is 
set, hardware registers cannot be accessed anyway, so skipping mutex and 
clearing local flag is safe.
+ - Canceling works before unregistering prevents concurrent execution during 
teardown, which is the desired behavior on unbind.
  
- [Notes]
- - Kernel taint "OE" comes only from VirtualBox modules (vboxdrv, vboxnetflt,
-   vboxnetadp). None of them appear in the traces.
- - Happens rarely. Other boots show no -110 errors and shut down normally.
- - I can test a -proposed / PPA kernel with the patches.
+ [ Other Info ]
+ Patch tested and compiled against linux-headers-7.0.0-31-generic on Ubuntu 
26.04 (Resolute). Clean compile with 0 errors and 0 warnings.
  
- ProblemType: Bug
- DistroRelease: Ubuntu 26.04
- Package: linux-image-7.0.0-31-generic 7.0.0-31.31
- ProcVersionSignature: Ubuntu 7.0.0-31.31-generic 7.0.14
- Uname: Linux 7.0.0-31-generic x86_64
- ApportVersion: 2.34.1-0ubuntu0.1
- Architecture: amd64
- AudioDevicesInUse:
-  USER        PID ACCESS COMMAND
-  /dev/snd/controlC1:  piotr      6207 F.... wireplumber
-  /dev/snd/controlC0:  piotr      6207 F.... wireplumber
-  /dev/snd/seq:        piotr      6192 F.... pipewire
- CasperMD5CheckResult: unknown
- CurrentDesktop: KDE
- Date: Thu Sep 17 15:39:30 2026
- InstallationDate: Installed on 2025-01-26 (599 days ago)
- InstallationMedia: Ubuntu-Studio 23.10 "Mantic Minotaur" - Release amd64 
(20231010)
- MachineType: Dell Inc. Precision 3650 Tower
- ProcFB: 0 amdgpudrmfb
- ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-7.0.0-31-generic 
root=UUID=d8c56c2f-379a-4ed5-8b74-f2e46feda11c ro quiet splash 
usbcore.autosuspend=-1 reboot=pci preempt=full rcu_nocbs=all preempt=full 
threadirqs rcu_nocbs=all
- PulseList: Error: command ['pacmd', 'list'] failed with exit code 1: No 
PulseAudio daemon running, or not running as session daemon.
- SourcePackage: linux
- UpgradeStatus: Upgraded to resolute on 2026-05-17 (123 days ago)
- dmi.bios.date: 05/27/2026
- dmi.bios.release: 1.48
- dmi.bios.vendor: Dell Inc.
- dmi.bios.version: 1.48.0
- dmi.board.name: 0NDYHG
- dmi.board.vendor: Dell Inc.
- dmi.board.version: A00
- dmi.chassis.type: 3
- dmi.chassis.vendor: Dell Inc.
- dmi.modalias: 
dmi:bvnDellInc.:bvr1.48.0:bd05/27/2026:br1.48:svnDellInc.:pnPrecision3650Tower:pvr:rvnDellInc.:rn0NDYHG:rvrA00:cvnDellInc.:ct3:cvr:sku0A58:pfaPrecision:
- dmi.product.family: Precision
- dmi.product.name: Precision 3650 Tower
- dmi.product.sku: 0A58
- dmi.sys.vendor: Dell Inc.
+ --- [ Original Report ]
+ Dell Precision 3650 Tower, Ubuntu 26.04 dev kernel 7.0.0-31-generic.
+ Alfa AWUS036AXM (MT7921AU, USB ID 0e8d:7961).
+ 
+ Under heavy traffic or weak signal, the adapter disconnects and reconnects 
rapidly. Eventually kernel dmesg shows:
+ [  342.112004] mt7921u 1-2:1.0: Message 00000040 (seq 4) timeout
+ [  345.184002] mt7921u 1-2:1.0: Message 00000040 (seq 5) timeout
+ [  348.256011] mt7921u 1-2:1.0: Failed to get patch sem
+ [  351.328008] mt7921u 1-2:1.0: hardware init failed
+ 
+ After that, NetworkManager stops responding. Running 'ip link' hangs 
indefinitely in D state.
+ Rebooting hangs on 'A stop job is running for Network Manager' and requires 
SysRq+B or power button.
+ 
+ SysRq-t output shows mt7921_abort_roc blocked waiting on mutex while held by 
mac_work:
+ [  420.100012] task:kworker/u16:3   blocked for more than 120 seconds.
+ [  420.100020] Call Trace:
+ [  420.100025]  __schedule+0x345/0x890
+ [  420.100030]  schedule+0x5a/0xc0
+ [  420.100035]  schedule_preempt_disabled+0x18/0x30
+ [  420.100040]  __mutex_lock.isra.0+0x28a/0x4b0
+ [  420.100045]  mt7921_abort_roc+0x2d/0x80 [mt7921_common]
+ [  420.100050]  ieee80211_set_disassoc+0x62/0x90 [cfg80211]
+ [  420.100055]  mt76_unregister_device+0x48/0x90 [mt76]
+ [  420.100060]  mt792xu_disconnect+0x3c/0x70 [mt792x_usb]

** Patch added: "Fix mt7921u/mt792xu USB disconnect deadlock on timeout"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2167595/+attachment/6001627/+files/lp2167595_mt7921u_usb_disconnect_deadlock.patch

** Changed in: linux (Ubuntu)
       Status: New => Confirmed

** Tags added: patch

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2167595

Title:
  mt7921u: USB reset after -110 timeouts deadlocks in mt7921_abort_roc,
  blocks rtnl_lock and shutdown

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2167595/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to