Sorry I am late to the party on confirming the fix, was traveling. Confirming that everything is working properly with the latest jammy- proposed kernel.
tested package: linux-image-5.15.0-1021-aws 5.15.0-1021.25 ubuntu@ip-172-31-31-116:~$ sudo dmesg | grep 5.15.60 [ 0.000000] Linux version 5.15.0-1021-aws (buildd@lcy02-amd64-074) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #25-Ubuntu SMP Fri Sep 23 12:20:42 UTC 2022 (Ubuntu 5.15.0-1021.25-aws 5.15.60) ubuntu@ip-172-31-31-116:~$ sudo dmesg | grep "zed amd" [ 5.170324] [drm] Initialized amdgpu 3.42.0 20150101 for 0000:00:1e.0 on minor 0 ubuntu@ip-172-31-31-116:~$ uname -a Linux ip-172-31-31-116 5.15.0-1021-aws #25-Ubuntu SMP Fri Sep 23 12:20:42 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux ubuntu@ip-172-31-31-116:~$ ls /dev/dri by-path card0 renderD128 ubuntu@ip-172-31-31-116:~$ apt list | grep linux- | grep installed | grep proposed WARNING: apt does not have a stable CLI interface. Use with caution in scripts. binutils-x86-64-linux-gnu/jammy-proposed,now 2.38-4ubuntu2 amd64 [installed,automatic] linux-aws-headers-5.15.0-1021/jammy-proposed,now 5.15.0-1021.25 all [installed,automatic] linux-aws/jammy-proposed,now 5.15.0.1021.21 amd64 [installed] linux-firmware/jammy-proposed,now 20220329.git681281e4-0ubuntu3.6 all [installed] linux-headers-5.15.0-1021-aws/jammy-proposed,now 5.15.0-1021.25 amd64 [installed,automatic] linux-headers-aws/jammy-proposed,now 5.15.0.1021.21 amd64 [installed] linux-image-5.15.0-1021-aws/jammy-proposed,now 5.15.0-1021.25 amd64 [installed,automatic] linux-image-aws/jammy-proposed,now 5.15.0.1021.21 amd64 [installed] linux-modules-5.15.0-1021-aws/jammy-proposed,now 5.15.0-1021.25 amd64 [installed,automatic] linux-modules-extra-5.15.0-1021-aws/jammy-proposed,now 5.15.0-1021.25 amd64 [installed,automatic] linux-modules-extra-aws/jammy-proposed,now 5.15.0.1021.21 amd64 [installed] ** Tags added: verification-done-jammy -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1981883 Title: amdgpu module crash after 5.15 kernel update Status in linux package in Ubuntu: Invalid Status in linux source package in Jammy: Fix Committed Bug description: [SRU Justification] Impact: The 5.15 kernel series introduced a regression compared to 5.13 related to virtual display code on AMD GPUs. This causes crashes for cloud/VM users back to Focal (using rolling cloud or HWE kernels). Fix: Upstream stable 5.15.61 contains a fix for this which is only needed there since later kernel version have that code rewritten again. This change would get picked up normally in the next stable cycle after 2022.09.19. But given the impact we would apply the following patch from 5.15.61 ahead of time: commit 27f8f5219fe4658537ba28fd01657e1062ac3960 linux-5.15.y "drm/amdgpu: fix check in fbdev init" Test case: Booting the affected kernel on a VM which passes through AMD GPU hardware. Regression potential: Worst case some driver functions will not get disabled where they did before. --- original description --- The kernel 5.15 amdgpu module crashes on load with a “BUG: kernel NULL pointer dereference” on Amazon EC2 G4ad hardware (custom AMD Radeon V520 Pro datacenter GPU) on focal (HWE) and jammy with kernel 5.15.0-1011 and possibly earlier, up through latest (revision 1015.19). This crash bug did not exist in any of the focal HWE 5.13 kernels. This is probably an upstream kernel bug, but I am also filing it here because existing focal users on EC2 will suddenly stop having access to their AMD GPUs after a reboot once the new 5.15 HWE kernel is installed. The full backtrace from dmesg is below. The offending function call which crashes in the 5.15 kernel corresponds to this source (sorry, not the right source tree, but the same driver) https://github.com/torvalds/linux/blob/8bb7eca972ad531c9b149c0a51ab43a417385813/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c#L345 A workaround that I have discovered is adding “options amdgpu virtual_display=;” to a new modprobe.d configuration file - something which shouldn’t be required, but is at least harmless. Here is the relevant BUG message and backtrace from dmesg: [ 318.111721] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 318.115443] #PF: supervisor instruction fetch in kernel mode [ 318.118443] #PF: error_code(0x0010) - not-present page [ 318.121177] PGD 0 P4D 0 [ 318.122688] Oops: 0010 [#1] SMP NOPTI [ 318.124592] CPU: 6 PID: 13667 Comm: modprobe Tainted: G W 5.15.0-1015-aws #19~20.04.1-Ubuntu [ 318.129711] Hardware name: Amazon EC2 g4ad.2xlarge/, BIOS 1.0 10/16/2017 [ 318.133167] RIP: 0010:0x0 [ 318.134704] Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6. [ 318.138291] RSP: 0018:ffff9841828d78e0 EFLAGS: 00010246 [ 318.140938] RAX: 0000000000000000 RBX: ffff8a4f16ae8000 RCX: 0000000000000001 [ 318.144604] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff8a4f16ae8000 [ 318.148319] RBP: ffff9841828d7908 R08: ffff8a4f02460278 R09: ffff8a4f06422c40 [ 318.152151] R10: c01c42494f8affff R11: ffff8a4f01dcb5b8 R12: ffff8a4f024602e8 [ 318.155929] R13: ffffffffc107e4a0 R14: 0000000000000000 R15: ffff8a4f02460010 [ 318.159685] FS: 00007f8afdc1c740(0000) GS:ffff8a55f0980000(0000) knlGS:0000000000000000 [ 318.163897] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 318.167038] CR2: ffffffffffffffd6 CR3: 000000011817e000 CR4: 00000000003506e0 [ 318.170758] Call Trace: [ 318.173964] <TASK> [ 318.176822] __drm_helper_disable_unused_functions+0xe7/0x100 [drm_kms_helper] [ 318.184230] drm_helper_disable_unused_functions+0x44/0x50 [drm_kms_helper] [ 318.189761] amdgpu_fbdev_init+0x104/0x110 [amdgpu] [ 318.194264] amdgpu_device_init.cold+0x7cc/0xc48 [amdgpu] [ 318.199061] ? pci_read_config_byte+0x27/0x40 [ 318.203206] amdgpu_driver_load_kms+0x1e/0x270 [amdgpu] [ 318.207901] amdgpu_pci_probe+0x1ea/0x290 [amdgpu] [ 318.212445] local_pci_probe+0x4b/0x90 [ 318.216386] pci_device_probe+0x182/0x1f0 [ 318.220407] really_probe.part.0+0xcb/0x370 [ 318.224460] really_probe+0x40/0x80 [ 318.228232] __driver_probe_device+0x115/0x190 [ 318.232412] driver_probe_device+0x23/0xa0 [ 318.236436] __driver_attach+0xbd/0x160 [ 318.240348] ? __device_attach_driver+0x110/0x110 [ 318.244637] bus_for_each_dev+0x7e/0xc0 [ 318.248570] driver_attach+0x1e/0x20 [ 318.252474] bus_add_driver+0x161/0x200 [ 318.256412] driver_register+0x74/0xd0 [ 318.260332] __pci_register_driver+0x68/0x70 [ 318.264496] amdgpu_init+0x7c/0x1000 [amdgpu] [ 318.268841] ? 0xffffffffc146e000 [ 318.272521] do_one_initcall+0x48/0x1d0 [ 318.276446] ? __cond_resched+0x19/0x30 [ 318.280378] ? kmem_cache_alloc_trace+0x15a/0x420 [ 318.284736] do_init_module+0x52/0x230 [ 318.288644] load_module+0x1372/0x1600 [ 318.292529] __do_sys_finit_module+0xbf/0x120 [ 318.296706] ? __do_sys_finit_module+0xbf/0x120 [ 318.300947] __x64_sys_finit_module+0x1a/0x20 [ 318.305265] do_syscall_64+0x5c/0xc0 [ 318.308984] ? do_syscall_64+0x69/0xc0 [ 318.312841] ? do_syscall_64+0x69/0xc0 [ 318.316693] ? do_syscall_64+0x69/0xc0 [ 318.320545] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 318.324975] RIP: 0033:0x7f8afdd6273d [ 318.328700] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 23 37 0d 00 f7 d8 64 89 01 48 [ 318.343601] RSP: 002b:00007ffe48e5c8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 318.351197] RAX: ffffffffffffffda RBX: 0000560c79e39260 RCX: 00007f8afdd6273d [ 318.356678] RDX: 0000000000000000 RSI: 0000560c78e96358 RDI: 000000000000000f [ 318.362340] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 318.367743] R10: 000000000000000f R11: 0000000000000246 R12: 0000560c78e96358 [ 318.373241] R13: 0000000000000000 R14: 0000560c79e39390 R15: 0000560c79e39260 [ 318.378699] </TASK> [ 318.381783] Modules linked in: amdgpu(+) iommu_v2 gpu_sched drm_ttm_helper ttm drm_kms_helper cec rc_core i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt btrfs blake2b_generic xor zstd_compress raid6_pq ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs libcrc32c nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ppdev crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel psmouse crypto_simd parport_pc input_leds parport cryptd ena serio_raw sch_fq_codel ipmi_devintf ipmi_msghandler msr drm ip_tables x_tables autofs4 [ 318.418449] CR2: 0000000000000000 [ 318.422130] ---[ end trace d6b9efffe55f5322 ]--- [ 318.426391] RIP: 0010:0x0 [ 318.429681] Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6. [ 318.434994] RSP: 0018:ffff9841828d78e0 EFLAGS: 00010246 [ 318.439489] RAX: 0000000000000000 RBX: ffff8a4f16ae8000 RCX: 0000000000000001 [ 318.444896] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff8a4f16ae8000 [ 318.450518] RBP: ffff9841828d7908 R08: ffff8a4f02460278 R09: ffff8a4f06422c40 [ 318.455937] R10: c01c42494f8affff R11: ffff8a4f01dcb5b8 R12: ffff8a4f024602e8 [ 318.461581] R13: ffffffffc107e4a0 R14: 0000000000000000 R15: ffff8a4f02460010 [ 318.466999] FS: 00007f8afdc1c740(0000) GS:ffff8a55f0980000(0000) knlGS:0000000000000000 [ 318.474744] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 318.479525] CR2: ffffffffffffffd6 CR3: 000000011817e000 CR4: 00000000003506e0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1981883/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
