Public bug reported: * Explain the bug(s)
We've found that for Jammy 5.15 and also 5.4.0-1049 kernel running on Bluefield, the kdump doesn't work when enabling secure boot[1]. * How to test Make sure kernel config: CONFIG_SECURITY_LOCKDOWN_LSM=y CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y CONFIG_LOCK_DOWN_IN_SECURE_BOOT=y The kdump kernel we use is actually signed correctly # file /boot/vmlinuz-5.15.0-1015-bluefield /boot/vmlinuz-5.15.0-1015-bluefield: gzip compressed data, was "vmlinuz-5.15.0 1015-bluefield.efi.signed", # kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0xcfe00000 /boot/vmlinuz-5.4.0-1049-bluefield kdump initrd: /boot/initrd.img-5.4.0-1049-bluefield current state: Not ready to kdump kdump-tools.service - Kernel crash dump capture service Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled) Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS) Main PID: 1975 (code=exited, status=0/SUCCESS) Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net systemd[1]: Starting Kernel crash dump capture serv> Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[1975]: Starting kdump-tools: Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu> Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu> Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2525]: syscall kexec_file_load not avai> Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * failed to load kdump kernel * Possible reason Currently, a problem faced by arm64 is if a kernel image is signed by a MOK key, loading it via the kexec_file_load() system call would be rejected with the error in dmesg "Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7". I backported the two below [2]: 0d519cadf751 arm64: kexec_file: use more system keyrings to verify kernel image signature c903dae8941d kexec, KEYS: make the code in bzImage64_verify_sig generic However still kdump/kexec fails due to lockdown [ 353.298348] Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7 [ 364.833004] audit: type=1400 audit(1686619435.331:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=15407 comm="apparmor_parser" If I disable kernel lockdown (CONFIG_SECURITY_LOCKDOWN_LSM=n), then kexec works ok. But this is not an acceptable workaround. * How to fix I got it working (secure boot + lockdown config enabled + kdump/kexec) using kernel v6.4 on bluefield, which means if we backport properly, mostly missing some arch/arm64/ patches, to BlueField 5.15 kernel, we can get it working. Reference 1. https://docs.nvidia.com/networking/display/BlueFieldDPUOSv392/UEFI%20Secure%20Boot 2. https://www.spinics.net/lists/arm-kernel/msg979554.html 3. https://mjg59.dreamwidth.org/50577.html ** Affects: linux-bluefield (Ubuntu) Importance: Undecided Status: New ** Summary changed: - kdump/kexec does not work when UEFI secureboot enabled + kdump/kexec does not work when UEFI secureboot and kernel lockdown enabled ** Description changed: * Explain the bug(s) We've found that for Jammy 5.15 and also 5.4.0-1049 kernel running on Bluefield, the kdump doesn't work when enabling secure boot[1]. * How to test + Make sure kernel config: + CONFIG_SECURITY_LOCKDOWN_LSM=y + CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y + CONFIG_LOCK_DOWN_IN_SECURE_BOOT=y + + The kdump kernel we use is actually signed correctly + # file /boot/vmlinuz-5.15.0-1015-bluefield + /boot/vmlinuz-5.15.0-1015-bluefield: gzip compressed data, was "vmlinuz-5.15.0 1015-bluefield.efi.signed", # kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0xcfe00000 - /boot/vmlinuz-5.4.0-1049-bluefield + /boot/vmlinuz-5.4.0-1049-bluefield kdump initrd: - /boot/initrd.img-5.4.0-1049-bluefield + /boot/initrd.img-5.4.0-1049-bluefield current state: Not ready to kdump kdump-tools.service - Kernel crash dump capture service - Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled) - Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago - Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS) - Main PID: 1975 (code=exited, status=0/SUCCESS) + Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled) + Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago + Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS) + Main PID: 1975 (code=exited, status=0/SUCCESS) Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net systemd[1]: Starting Kernel crash dump capture serv> Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[1975]: Starting kdump-tools: Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu> Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu> Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2525]: syscall kexec_file_load not avai> Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * failed to load kdump kernel * Possible reason Currently, a problem faced by arm64 is if a kernel image is signed by a MOK key, loading it via the kexec_file_load() system call would be rejected with the error in dmesg "Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7". I backported the two below [2]: 0d519cadf751 arm64: kexec_file: use more system keyrings to verify kernel image signature c903dae8941d kexec, KEYS: make the code in bzImage64_verify_sig generic However still kdump/kexec fails due to lockdown [ 353.298348] Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7 [ 364.833004] audit: type=1400 audit(1686619435.331:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=15407 comm="apparmor_parser" If I disable kernel lockdown (CONFIG_SECURITY_LOCKDOWN_LSM=n), then kexec works ok. But this is not an acceptable workaround. * How to fix I got it working (secure boot + lockdown config enabled + kdump/kexec) using kernel v6.4 on bluefield, which means if we backport properly, mostly missing some arch/arm64/ patches, to BlueField 5.15 kernel, we can get it working. - Reference 1. https://docs.nvidia.com/networking/display/BlueFieldDPUOSv392/UEFI%20Secure%20Boot 2. https://www.spinics.net/lists/arm-kernel/msg979554.html 3. https://mjg59.dreamwidth.org/50577.html -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-bluefield in Ubuntu. https://bugs.launchpad.net/bugs/2025396 Title: kdump/kexec does not work when UEFI secureboot and kernel lockdown enabled Status in linux-bluefield package in Ubuntu: New Bug description: * Explain the bug(s) We've found that for Jammy 5.15 and also 5.4.0-1049 kernel running on Bluefield, the kdump doesn't work when enabling secure boot[1]. * How to test Make sure kernel config: CONFIG_SECURITY_LOCKDOWN_LSM=y CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y CONFIG_LOCK_DOWN_IN_SECURE_BOOT=y The kdump kernel we use is actually signed correctly # file /boot/vmlinuz-5.15.0-1015-bluefield /boot/vmlinuz-5.15.0-1015-bluefield: gzip compressed data, was "vmlinuz-5.15.0 1015-bluefield.efi.signed", # kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0xcfe00000 /boot/vmlinuz-5.4.0-1049-bluefield kdump initrd: /boot/initrd.img-5.4.0-1049-bluefield current state: Not ready to kdump kdump-tools.service - Kernel crash dump capture service Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled) Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS) Main PID: 1975 (code=exited, status=0/SUCCESS) Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net systemd[1]: Starting Kernel crash dump capture serv> Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[1975]: Starting kdump-tools: Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu> Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu> Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2525]: syscall kexec_file_load not avai> Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * failed to load kdump kernel * Possible reason Currently, a problem faced by arm64 is if a kernel image is signed by a MOK key, loading it via the kexec_file_load() system call would be rejected with the error in dmesg "Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7". I backported the two below [2]: 0d519cadf751 arm64: kexec_file: use more system keyrings to verify kernel image signature c903dae8941d kexec, KEYS: make the code in bzImage64_verify_sig generic However still kdump/kexec fails due to lockdown [ 353.298348] Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7 [ 364.833004] audit: type=1400 audit(1686619435.331:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=15407 comm="apparmor_parser" If I disable kernel lockdown (CONFIG_SECURITY_LOCKDOWN_LSM=n), then kexec works ok. But this is not an acceptable workaround. * How to fix I got it working (secure boot + lockdown config enabled + kdump/kexec) using kernel v6.4 on bluefield, which means if we backport properly, mostly missing some arch/arm64/ patches, to BlueField 5.15 kernel, we can get it working. Reference 1. https://docs.nvidia.com/networking/display/BlueFieldDPUOSv392/UEFI%20Secure%20Boot 2. https://www.spinics.net/lists/arm-kernel/msg979554.html 3. https://mjg59.dreamwidth.org/50577.html To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-bluefield/+bug/2025396/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
