This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.
If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1876707 Title: NULL pointer dereference in nvme reset work-queue when VMD raid mode and SecureBoot turned on simultaneously on TigerLake Status in HWE Next: New Status in linux package in Ubuntu: In Progress Status in linux-oem-5.6 package in Ubuntu: Invalid Status in linux source package in Focal: Invalid Status in linux-oem-5.6 source package in Focal: Fix Committed Status in linux source package in Groovy: In Progress Status in linux-oem-5.6 source package in Groovy: Invalid Bug description: [SRU Justfication] [Impact] On platforms with NVMe attached to VMD controller, enable SecureBoot would also force enable iommu: DMAR: Intel-IOMMU force enabled due to platform opt in While devices behind the VMD controller, also a PCI bridge, maybe forced to use a DMA domain by current Intel-IOMMU driver, this may break some relationships between sub devices behind VMD controller and between the VMD controller and its children devices, and finally caused undefined system behavior. On devices at hand, this results in kernel NULL dereference at __intel_map_single called from nvme_reset_work and fails root device lookup at boot. kernel: BUG: kernel NULL pointer dereference, address: 0000000000000018 kernel: #PF: supervisor read access in kernel mode kernel: #PF: error_code(0x0000) - not-present page kernel: PGD 0 P4D 0 kernel: Oops: 0000 [#2] SMP NOPTI kernel: CPU: 1 PID: 254 Comm: kworker/u8:4 Tainted: G D W 5.7.0-050700rc3-generic #202004262131 kernel: Hardware name: Dell Inc. Vostro 5402/, BIOS 0.1.2 04/13/2020 kernel: Workqueue: nvme-reset-wq nvme_reset_work [nvme] kernel: RIP: 0010:__intel_map_single+0xa3/0x1a0 ... [Fix] Patchset[1] currently landed in iommu/next beginning with commit 327d5b2fee91 ("iommu/vt-d: Allow 32bit devices to uses DMA domain") gives the solution to this problem. However, it's based on a massive subsystem rewrite in patchset[2], currently in iommu/next beginning with commit 441ff2ff8327 ("Move default domain allocation to separate function"). On v5.6, it also depends on yet a few more patch series landed in v5.7-rc1 beginning with commit 098accf2da94 ("iommu: Use C99 flexible array in fwspec") that rewrote private data access, changed struct names, etc. Yet a few additional patches included as fixes to above changes. [1]: https://lore.kernel.org/linux-iommu/7928dd48-93da-62f0-b455-6e6b248d0...@linux.intel.com/ [2]: https://lore.kernel.org/linux-iommu/20200429133712.31431-1-j...@8bytes.org/ [Test Case] Test on platforms with VMD/NVMe and enable SecureBoot. System should boot normally rather than into initramfs emergency shell. [Regression Potential] For unstable, all the patches are from iommu-next and will probably be merged in next few -rc releases, so should be safe to place a LOW here. For oem-5.6, the fixing patchset is depending on iommu group setup refactoring that touched almost every architecture/platform uses iommu although we would only care amd64 among them. Even with follow-up fixes included, this is still a 60-patches change and deserves some more attention. Medium. ========== Original Bug Description ========== This is found on a Dell TigerLake platform that when VMD raid mode is turned on along with SecureBoot, either deploy mode or audit mode, kernel dumps warnings and null pointer deref errors at boot. While it happens, it blocks systemd-udevd worker processes until killed due to timeout. System still boots to multi-users.target. Kernel bisect shows commit e3560ee4cfb2 ("iommu/vt-d: Remove VMD child device sanity check") merged in v5.6-rc1 is the first commit to fail, and is still reproducible on v5.7-rc3. kernel: Secure boot disabled ... kernel: ------------[ cut here ]------------ kernel: WARNING: CPU: 1 PID: 8 at drivers/iommu/intel-iommu.c:625 domain_get_iommu+0x4b/0x60 kernel: Modules linked in: rc_core r8169(+) intel_lpss nvme crc32_pclmul(+) psmouse intel_ish_ipc(+) i2c_hid i2c_i801(+) realtek idma64 drm virt_dma intel_ishtp vmd(+) nvme_core hid video wmi pinctrl_tigerlake pinctrl_intel kernel: CPU: 1 PID: 8 Comm: kworker/u8:0 Not tainted 5.7.0-050700rc3-generic #202004262131 kernel: Hardware name: Dell Inc. Vostro 5402/, BIOS 0.1.2 04/13/2020 kernel: Workqueue: nvme-reset-wq nvme_reset_work [nvme] kernel: RIP: 0010:domain_get_iommu+0x4b/0x60 kernel: Code: eb 22 48 8d 50 01 48 39 c8 74 1b 48 89 d0 8b 74 87 04 48 63 d0 85 f6 74 e9 48 8b 05 ef 63 63 01 48 8b 04 d0 5d c3 31 c0 5d c3 <0f> 0b 31 c0 5d c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f kernel: RSP: 0018:ffffb5f5c00fbcf8 EFLAGS: 00010202 kernel: RAX: ffff9ebdaf100b00 RBX: 0000000000000000 RCX: 0000000000000000 kernel: RDX: 0000000000001000 RSI: 000000036dd41000 RDI: ffff9ebdaf100b00 kernel: RBP: ffffb5f5c00fbcf8 R08: ffffffffffffffff R09: ffff9ebdadd41000 kernel: R10: ffffffff8d069060 R11: 0000000000004879 R12: ffff9ebdadd810b0 kernel: R13: 000000036dd41000 R14: ffffffffffffffff R15: ffff9ebdaf100b00 kernel: FS: 0000000000000000(0000) GS:ffff9ebdc1680000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 00007f900d20e660 CR3: 000000036e05a003 CR4: 0000000000760ee0 kernel: PKRU: 55555554 kernel: Call Trace: kernel: __intel_map_single+0x47/0x1a0 kernel: intel_alloc_coherent+0xab/0x120 kernel: dma_alloc_attrs+0x4d/0x60 kernel: nvme_alloc_queue+0x63/0x180 [nvme] kernel: nvme_reset_work+0x31a/0xa64 [nvme] kernel: ? wake_up_process+0x15/0x20 kernel: ? swake_up_locked.part.0+0x17/0x30 kernel: process_one_work+0x1e8/0x3b0 kernel: worker_thread+0x4d/0x400 kernel: kthread+0x104/0x140 kernel: ? process_one_work+0x3b0/0x3b0 kernel: ? kthread_park+0x90/0x90 kernel: ret_from_fork+0x1f/0x40 kernel: ---[ end trace caf06459a58aa8d4 ]--- .... kernel: BUG: kernel NULL pointer dereference, address: 0000000000000018 kernel: #PF: supervisor read access in kernel mode kernel: #PF: error_code(0x0000) - not-present page kernel: PGD 0 P4D 0 kernel: Oops: 0000 [#2] SMP NOPTI kernel: CPU: 1 PID: 254 Comm: kworker/u8:4 Tainted: G D W 5.7.0-050700rc3-generic #202004262131 kernel: Hardware name: Dell Inc. Vostro 5402/, BIOS 0.1.2 04/13/2020 kernel: Workqueue: nvme-reset-wq nvme_reset_work [nvme] kernel: RIP: 0010:__intel_map_single+0xa3/0x1a0 kernel: Code: 89 d2 4c 89 55 d0 e8 ec b3 ff ff 4c 8b 55 d0 48 85 c0 49 89 c6 0f 84 e9 00 00 00 41 b9 01 00 00 00 83 fb 01 76 14 48 8b 45 c0 <4c> 8b 48 18 49 c1 e9 16 49 83 f1 01 41 83 e1 01 44 89 c8 4c 89 e9 kernel: RSP: 0018:ffffb5f5c050f878 EFLAGS: 00010202 kernel: RAX: 0000000000000000 RBX: 0000000000000002 RCX: ffff9ebdbf205140 kernel: RDX: ffff9ebdbf205bc0 RSI: 0000000000000257 RDI: ffff9ebdaf100e30 kernel: RBP: ffffb5f5c050f8c0 R08: ffff9ebdae266f00 R09: 0000000000000001 kernel: R10: 0000000000000001 R11: 0000000000000022 R12: ffff9ebdadd860b0 kernel: R13: 000000036ef0d000 R14: 00000000000ffffa R15: ffff9ebdaf100b00 kernel: FS: 0000000000000000(0000) GS:ffff9ebdc1680000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 0000000000000018 CR3: 000000036efd2001 CR4: 0000000000760ee0 kernel: PKRU: 55555554 kernel: Call Trace: kernel: intel_map_page+0x86/0xa0 kernel: nvme_map_data+0x486/0x990 [nvme] kernel: ? fbcon_cursor+0x128/0x180 kernel: ? bit_putcs+0x5a0/0x5a0 kernel: nvme_queue_rq+0xa2/0x1d0 [nvme] kernel: blk_mq_dispatch_rq_list+0x93/0x5d0 kernel: ? __alloc_pages_nodemask+0x161/0x2f0 kernel: ? _find_next_bit.constprop.0+0x20/0x80 kernel: blk_mq_sched_dispatch_requests+0xfe/0x180 kernel: __blk_mq_run_hw_queue+0x5a/0x110 kernel: __blk_mq_delay_run_hw_queue+0x15b/0x160 kernel: blk_mq_run_hw_queue+0x70/0x110 kernel: blk_mq_sched_insert_request+0xce/0x190 kernel: ? blk_rq_append_bio+0x28/0x180 kernel: blk_execute_rq_nowait+0x61/0x70 kernel: blk_execute_rq+0x50/0xb0 kernel: __nvme_submit_sync_cmd+0x92/0x1e0 [nvme_core] kernel: ? __cpuhp_state_add_instance_cpuslocked+0xe8/0x110 kernel: nvme_identify_ctrl.isra.0+0x7e/0xc0 [nvme_core] kernel: nvme_init_identify+0x97/0x6d0 [nvme_core] kernel: nvme_reset_work+0x422/0xa64 [nvme] kernel: ? try_to_wake_up+0x65/0x690 kernel: process_one_work+0x1e8/0x3b0 kernel: worker_thread+0x4d/0x400 kernel: kthread+0x104/0x140 kernel: ? process_one_work+0x3b0/0x3b0 kernel: ? kthread_park+0x90/0x90 kernel: ret_from_fork+0x1f/0x40 kernel: Modules linked in: cec(+) intel_lpss_pci(+) rc_core fjes(-) r8169(+) intel_lpss nvme crc32_pclmul psmouse intel_ish_ipc(+) i2c_hid i2c_i801(+) realtek idma64 drm virt_dma intel_ishtp vmd nvme_core hid video wmi pinctrl_tigerlake pinctrl_intel kernel: CR2: 0000000000000018 kernel: ---[ end trace caf06459a58aa8db ]--- kernel: RIP: 0010:__intel_map_single+0xa3/0x1a0 kernel: Code: 89 d2 4c 89 55 d0 e8 ec b3 ff ff 4c 8b 55 d0 48 85 c0 49 89 c6 0f 84 e9 00 00 00 41 b9 01 00 00 00 83 fb 01 76 14 48 8b 45 c0 <4c> 8b 48 18 49 c1 e9 16 49 83 f1 01 41 83 e1 01 44 89 c8 4c 89 e9 kernel: RSP: 0018:ffffb5f5c00fb878 EFLAGS: 00010202 kernel: RAX: 0000000000000000 RBX: 0000000000000002 RCX: ffff9ebdae2665c0 kernel: RDX: ffff9ebdbf205581 RSI: 0000000000000257 RDI: ffff9ebdaf100e30 kernel: RBP: ffffb5f5c00fb8c0 R08: ffff9ebdaf100e38 R09: 0000000000000001 kernel: R10: 0000000000000001 R11: 0000000000000022 R12: ffff9ebdadd810b0 kernel: R13: 000000036ef0e000 R14: 00000000000ffffd R15: ffff9ebdaf100b00 kernel: FS: 0000000000000000(0000) GS:ffff9ebdc1680000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 0000000000000018 CR3: 000000036efd2001 CR4: 0000000000760ee0 kernel: PKRU: 55555554 To manage notifications about this bug go to: https://bugs.launchpad.net/hwe-next/+bug/1876707/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
