Hi Vadik, Oliver, Thanks for reporting, and sorry that 5.13.0-24-generic in -proposed didn't solve the issue.
Let's do some analysis: [ 1.381250] BUG: kernel NULL pointer dereference, address: 000000000000000c [ 1.381270] RIP: 0010:amd_sfh_hid_client_init+0x47/0x350 [amd_sfh] [ 1.381299] Call Trace: [ 1.381302] ? __pci_set_master+0x5f/0xe0 [ 1.381310] amd_mp2_pci_probe+0xad/0x160 [amd_sfh] [ 1.381314] local_pci_probe+0x48/0x80 ... Okay, so a null pointer dereference in the amd_sfh module. The c in 000000000000000c probably means offset +12 in the struct we are trying to access. Let's see where this is: $ eu-addr2line -ifae ./usr/lib/debug/lib/modules/5.13.0-23-generic/kernel/drivers/hid/amd-sfh-hid/amd_sfh.ko amd_sfh_hid_client_init+0x47 0x0000000000000767 amd_sfh_hid_client_init /build/linux-k2e9CH/linux-5.13.0/drivers/hid/amd-sfh-hid/amd_sfh_client.c:147:27 Let's have a look: 134 int amd_sfh_hid_client_init(struct amd_mp2_dev *privdata) 135 { ... 146 147 cl_data->num_hid_devices = amd_mp2_get_sensor_num(privdata, &cl_data->sensor_idx[0]); 148 ... Okay, so we are dereferencing either cl_data->num_hid_devices or &cl_data->sensor_idx[0], but they are both in cl_data, so cl_data will be NULL. Since you mentioned that it worked in 5.13.0-22-generic, and broke in 5.13.0-23-generic, lets see if this changed in 5.13.0-23-generic: $ git log --grep "amd_sfh" Ubuntu-5.13.0-22.22..Ubuntu-5.13.0-23.23 commit d46ef750ed58cbeeba2d9a55c99231c30a172764 commit-impish 56559d7910e704470ad72da58469b5588e8cbf85 Author: Evgeny Novikov <novi...@ispras.ru> Date: Tue Jun 1 19:38:01 2021 +0300 Subject:HID: amd_sfh: Fix potential NULL pointer dereference Link: https://github.com/torvalds/linux/commit/d46ef750ed58cbeeba2d9a55c99231c30a172764 Okay, so this patch changes the parent function to amd_sfh_hid_client_init(), which is amd_mp2_pci_probe(). + rc = amd_sfh_hid_client_init(privdata); + if (rc) + return rc; + privdata->cl_data = devm_kzalloc(&pdev->dev, sizeof(struct amdtp_cl_data), GFP_KERNEL); if (!privdata->cl_data) return -ENOMEM; ... - return amd_sfh_hid_client_init(privdata); + return 0; So it seems we are moving the call to amd_sfh_hid_client_init(privdata) from the end of the function up a bit, and interestingly, before the call to privdata->cl_data = devm_kzalloc(). So... we are using privdata->cl_data before it is being allocated? Looks like we have found our NULL pointer dereference. I suppose the commit to "fix" the null pointer dereference actually introduced another one. Looking at this commit in the upstream tree, I came across: commit 88a04049c08cd62e698bc1b1af2d09574b9e0aee Author: Basavaraj Natikar <basavaraj.nati...@amd.com> Date: Thu Sep 23 17:59:27 2021 +0530 Subject: HID: amd_sfh: Fix potential NULL pointer dereference Link: https://github.com/torvalds/linux/commit/88a04049c08cd62e698bc1b1af2d09574b9e0aee This patch seems to move the call to after cl_data is allocated, which should fix this. - rc = amd_sfh_hid_client_init(privdata); - if (rc) - return rc; - privdata->cl_data = devm_kzalloc(&pdev->dev, sizeof(struct amdtp_cl_data), GFP_KERNEL); if (!privdata->cl_data) return -ENOMEM; - rc = devm_add_action_or_reset(&pdev->dev, amd_mp2_pci_remove, privdata); + mp2_select_ops(privdata); + + rc = amd_sfh_hid_client_init(privdata); This commit landed in 5.15-rc4: $ git describe --contains 88a04049c08cd62e698bc1b1af2d09574b9e0aee v5.15-rc4~40^2 It seems it was backported to 5.14.10: https://lwn.net/Articles/872195/ Impish should have gotten 5.14.10 during its regular upstream -stable patches: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1950388 The commit is listed there, but when I search the Impish git tree, it is missing? I think what has happened is the two commits have the same name, and Kamal must have gotten confused and thought it was a duplicate, and dropped it. Here's what we are going to do. I will build you a test kernel based on 5.13.0-23-generic, that includes Basavaraj Natikar's fix, and I will provide instructions on how to install it. You can test it to make sure it fixes the issue, and if it does, I will submit the patch for SRU to the 5.13 kernel. I will write back once the test kernel has finished building, probably tomorrow. Thanks, Matthew ** Changed in: linux (Ubuntu Impish) Assignee: (unassigned) => Matthew Ruffell (mruffell) ** Tags added: seg -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1956519 Title: kernel panic after upgrading to kernel 5.13.0-23 Status in linux package in Ubuntu: Fix Released Status in linux source package in Impish: In Progress Bug description: After upgrading my son's Asus PN50 with Ubuntu 21.10 to the latest kernel 5.13.0-23, I am no longer able to boot it normally. Kernel fails with the panic halfway through the boot process (which got overall suspiciously slow): [ 1.359465] BUG: kernel NULL pointer dereference, address: 000000000000000c [ 1.359498] #PF: supervisor write access in kernel mode [ 1.359519] #PF: error_code(0x0002) - not-present page [ 1.359540] PGD 0 P4D 0 [ 1.359553] Oops: 0002 [#1] SMP NOPTI [ 1.359569] CPU: 0 PID: 175 Comm: systemd-udevd Not tainted 5.13.0-23-generic #23-Ubuntu [ 1.359602] Hardware name: ASUSTeK COMPUTER INC. MINIPC PN50/PN50, BIOS 0623 05/13/2021 [ 1.359632] RIP: 0010:amd_sfh_hid_client_init+0x47/0x350 [amd_sfh] [ 1.359661] Code: 00 53 48 83 ec 20 48 8b 5f 08 48 8b 07 48 8d b3 22 01 00 00 4c 8d b0 c8 00 00 00 e8 23 07 00 00 45 31 c0 31 c9 ba 00 00 20 00 <89> 43 0c 48 8d 83 68 01 00 00 48 8d bb 80 01 00 00 48 c7 c6 20 6d [ 1.359729] RSP: 0018:ffffbf71c099f9d8 EFLAGS: 00010246 [ 1.359750] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 1.359777] RDX: 0000000000200000 RSI: ffffffffc03cd249 RDI: ffffffffa680004c [ 1.359804] RBP: ffffbf71c099fa20 R08: 0000000000000000 R09: 0000000000000006 [ 1.359831] R10: ffffbf71c0d00000 R11: 0000000000000007 R12: 0000000fffffffe0 [ 1.359857] R13: ffff992bc3387cd8 R14: ffff992bc11560c8 R15: ffff992bc3387cd8 [ 1.359884] FS: 00007ff0ec1a48c0(0000) GS:ffff992ebf600000(0000) knlGS:0000000000000000 [ 1.359915] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1.359937] CR2: 000000000000000c CR3: 0000000102fd0000 CR4: 0000000000350ef0 [ 1.359964] Call Trace: [ 1.359976] ? __pci_set_master+0x5f/0xe0 [ 1.359997] amd_mp2_pci_probe+0xad/0x160 [amd_sfh] [ 1.360021] local_pci_probe+0x48/0x80 [ 1.360038] pci_device_probe+0x105/0x1c0 [ 1.360056] really_probe+0x24b/0x4c0 [ 1.360073] driver_probe_device+0xf0/0x160 [ 1.360091] device_driver_attach+0xab/0xb0 [ 1.360110] __driver_attach+0xb2/0x140 [ 1.360126] ? device_driver_attach+0xb0/0xb0 [ 1.360145] bus_for_each_dev+0x7e/0xc0 [ 1.360161] driver_attach+0x1e/0x20 [ 1.360177] bus_add_driver+0x135/0x1f0 [ 1.360194] driver_register+0x95/0xf0 [ 1.360210] ? 0xffffffffc03d2000 [ 1.360225] __pci_register_driver+0x57/0x60 [ 1.360242] amd_mp2_pci_driver_init+0x23/0x1000 [amd_sfh] [ 1.360266] do_one_initcall+0x48/0x1d0 [ 1.360284] ? kmem_cache_alloc_trace+0xfb/0x240 [ 1.360306] do_init_module+0x62/0x290 [ 1.360323] load_module+0xa8f/0xb10 [ 1.360340] __do_sys_finit_module+0xc2/0x120 [ 1.360359] __x64_sys_finit_module+0x18/0x20 [ 1.360377] do_syscall_64+0x61/0xb0 [ 1.361638] ? ksys_mmap_pgoff+0x135/0x260 [ 1.362883] ? exit_to_user_mode_prepare+0x37/0xb0 [ 1.364121] ? syscall_exit_to_user_mode+0x27/0x50 [ 1.365343] ? __x64_sys_mmap+0x33/0x40 [ 1.366550] ? do_syscall_64+0x6e/0xb0 [ 1.367749] ? do_syscall_64+0x6e/0xb0 [ 1.368923] ? do_syscall_64+0x6e/0xb0 [ 1.370079] ? syscall_exit_to_user_mode+0x27/0x50 [ 1.371227] ? do_syscall_64+0x6e/0xb0 [ 1.372359] ? exc_page_fault+0x8f/0x170 [ 1.373478] ? asm_exc_page_fault+0x8/0x30 [ 1.374584] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 1.375684] RIP: 0033:0x7ff0ec73a94d [ 1.376767] Code: 5b 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d b3 64 0f 00 f7 d8 64 89 01 48 [ 1.377926] RSP: 002b:00007ffd00724ba8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 1.379076] RAX: ffffffffffffffda RBX: 000055e130084390 RCX: 00007ff0ec73a94d [ 1.380225] RDX: 0000000000000000 RSI: 00007ff0ec8ca3fe RDI: 0000000000000005 [ 1.381363] RBP: 0000000000020000 R08: 0000000000000000 R09: 0000000000000000 [ 1.382488] R10: 0000000000000005 R11: 0000000000000246 R12: 00007ff0ec8ca3fe [ 1.383598] R13: 000055e130083370 R14: 000055e130084480 R15: 000055e130086cb0 [ 1.384698] Modules linked in: ahci(+) libahci i2c_piix4(+) r8169(+) amd_sfh(+) i2c_hid_acpi realtek i2c_hid xhci_pci(+) xhci_pci_renesas wmi(+) video(+) fjes(+) hid [ 1.385841] CR2: 000000000000000c [ 1.386955] ---[ end trace b2ebcacf74b788da ]--- [ 1.388064] RIP: 0010:amd_sfh_hid_client_init+0x47/0x350 [amd_sfh] [ 1.389176] Code: 00 53 48 83 ec 20 48 8b 5f 08 48 8b 07 48 8d b3 22 01 00 00 4c 8d b0 c8 00 00 00 e8 23 07 00 00 45 31 c0 31 c9 ba 00 00 20 00 <89> 43 0c 48 8d 83 68 01 00 00 48 8d bb 80 01 00 00 48 c7 c6 20 6d [ 1.390374] RSP: 0018:ffffbf71c099f9d8 EFLAGS: 00010246 [ 1.391560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 1.392338] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20 [ 1.392763] RDX: 0000000000200000 RSI: ffffffffc03cd249 RDI: ffffffffa680004c [ 1.395162] RBP: ffffbf71c099fa20 R08: 0000000000000000 R09: 0000000000000006 [ 1.396372] R10: ffffbf71c0d00000 R11: 0000000000000007 R12: 0000000fffffffe0 [ 1.397564] R13: ffff992bc3387cd8 R14: ffff992bc11560c8 R15: ffff992bc3387cd8 [ 1.398754] FS: 00007ff0ec1a48c0(0000) GS:ffff992ebf600000(0000) knlGS:0000000000000000 [ 1.399916] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1.401044] CR2: 000000000000000c CR3: 0000000102fd0000 CR4: 0000000000350ef0 Previous kernel 5.13.0-22 works alright. ProblemType: Bug DistroRelease: Ubuntu 21.10 Package: linux-image-5.13.0-23-generic 5.13.0-23.23 ProcVersionSignature: Ubuntu 5.13.0-22.22-generic 5.13.19 Uname: Linux 5.13.0-22-generic x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair ApportVersion: 2.20.11-0ubuntu71 Architecture: amd64 AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-id', '/dev/snd/controlC1', '/dev/snd/pcmC1D0c', '/dev/snd/controlC2', '/dev/snd/hwC2D0', '/dev/snd/pcmC2D0c', '/dev/snd/pcmC2D0p', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D9p', '/dev/snd/pcmC0D8p', '/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CasperMD5CheckResult: unknown Date: Wed Jan 5 19:00:15 2022 InstallationDate: Installed on 2021-01-01 (369 days ago) InstallationMedia: Ubuntu 20.10 "Groovy Gorilla" - Release amd64 (20201022) MachineType: ASUSTeK COMPUTER INC. MINIPC PN50 ProcFB: 0 amdgpudrmfb ProcKernelCmdLine: BOOT_IMAGE=/BOOT/ubuntu_ct91lc@/vmlinuz-5.13.0-22-generic root=ZFS=rpool/ROOT/ubuntu_ct91lc ro quiet splash RelatedPackageVersions: linux-restricted-modules-5.13.0-22-generic N/A linux-backports-modules-5.13.0-22-generic N/A linux-firmware 1.201.3 SourcePackage: linux UpgradeStatus: Upgraded to impish on 2021-10-17 (80 days ago) WifiSyslog: dmi.bios.date: 05/13/2021 dmi.bios.release: 6.23 dmi.bios.vendor: ASUSTeK COMPUTER INC. dmi.bios.version: 0623 dmi.board.asset.tag: Default string dmi.board.name: PN50 dmi.board.vendor: ASUSTeK COMPUTER INC. dmi.board.version: To be filled by O.E.M. dmi.chassis.asset.tag: Default string dmi.chassis.type: 35 dmi.chassis.vendor: Default string dmi.chassis.version: Default string dmi.modalias: dmi:bvnASUSTeKCOMPUTERINC.:bvr0623:bd05/13/2021:br6.23:svnASUSTeKCOMPUTERINC.:pnMINIPCPN50:pvr0623:rvnASUSTeKCOMPUTERINC.:rnPN50:rvrTobefilledbyO.E.M.:cvnDefaultstring:ct35:cvrDefaultstring:sku: dmi.product.family: Vivo PC dmi.product.name: MINIPC PN50 dmi.product.version: 0623 dmi.sys.vendor: ASUSTeK COMPUTER INC. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1956519/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp