** Description changed: # Overview Hirsute and Impish daily builds are currently not booting on Google Confidential Compute. Confidential compute is Google's platform that enables the use of Secure Encrypted Virtualization extension via AMD EPYC CPUs. Booting an image with version 1.45 works, but once upgraded to 1.47, the VM no longer boots, and instead the kernel panics. Launching the image with secure boot, but without confidential compute works as expected. # Expected result The system is able to reboot after the upgrade. # Actual result Kernel panic: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Steps to reproduce Launch a VM in GCE with confidential compute enabled with a serial v20210511a or later and look at the serial log for the kernel panic. Example CLI command to launch a VM: $ gcloud beta compute instances create $USER-confidential-testing --zone=us-west1-b --machine-type=n2d-standard-2 --image=daily- ubuntu-2104-hirsute-v20210511a --image-project=ubuntu-os-cloud-devel --confidential-compute --maintenance-policy=TERMINATE The last known good working image is daily- ubuntu-2104-hirsute-v20210510. The upgrade that fails is when shim signed is updated from 1.46+15.4-0ubuntu1 to 1.47+15.4-0ubuntu2 # Logs & notes * 20210510 manifest (good): https://paste.ubuntu.com/p/QjnMPcJj7G/ * 20210511a manifest (bad): https://paste.ubuntu.com/p/PvJQwRXHcG/ * diff between manifests: https://paste.ubuntu.com/p/4nJtGxqGn7/ * serial logs of failed boot: https://paste.ubuntu.com/p/mHrvVc6qBc/ + + + # Cause: + + shim changed the memory type for pages reserved for EFI runtime + services, from EfiRuntimeServicesData to EfiBootServicesData. + + Memory reserved for EFI runtime/boot services must be remapped as + encrypted in the kernel (during boot) if SEV (secure encrypted + virtualization) is enabled. The original kernel implementation of + ioremap only correctly mapped the region as encrypted for + EfiRuntimeServicesData regions, so when shim changed the type to + EfiBootServicesData the kernel bug was exposed + + # Fix: + + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8d651ee9c71bb12fc0c8eb2786b66cbe5aa3e43b + + + # Regression potential + + The fix could potentially cause boot failures, if a memory region is + marked encrypted when it isn't. I assume it would cause a panic similar + to the one seen here for this bug: + + general protection fault, probably for non-canonical address + 0x314836c31124d346: 0000 [#1] SMP NOPTI
** Description changed: # Overview Hirsute and Impish daily builds are currently not booting on Google Confidential Compute. Confidential compute is Google's platform that enables the use of Secure Encrypted Virtualization extension via AMD EPYC CPUs. Booting an image with version 1.45 works, but once upgraded to 1.47, the VM no longer boots, and instead the kernel panics. Launching the image with secure boot, but without confidential compute works as expected. # Expected result The system is able to reboot after the upgrade. # Actual result Kernel panic: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Steps to reproduce Launch a VM in GCE with confidential compute enabled with a serial v20210511a or later and look at the serial log for the kernel panic. Example CLI command to launch a VM: $ gcloud beta compute instances create $USER-confidential-testing --zone =us-west1-b --machine-type=n2d-standard-2 --image=daily-ubuntu-2104 -hirsute-v20210511a --image-project=ubuntu-os-cloud-devel --confidential-compute --maintenance-policy=TERMINATE The last known good working image is daily- ubuntu-2104-hirsute-v20210510. The upgrade that fails is when shim signed is updated from 1.46+15.4-0ubuntu1 to 1.47+15.4-0ubuntu2 # Logs & notes * 20210510 manifest (good): https://paste.ubuntu.com/p/QjnMPcJj7G/ * 20210511a manifest (bad): https://paste.ubuntu.com/p/PvJQwRXHcG/ * diff between manifests: https://paste.ubuntu.com/p/4nJtGxqGn7/ * serial logs of failed boot: https://paste.ubuntu.com/p/mHrvVc6qBc/ - # Cause: shim changed the memory type for pages reserved for EFI runtime services, from EfiRuntimeServicesData to EfiBootServicesData. Memory reserved for EFI runtime/boot services must be remapped as encrypted in the kernel (during boot) if SEV (secure encrypted virtualization) is enabled. The original kernel implementation of ioremap only correctly mapped the region as encrypted for EfiRuntimeServicesData regions, so when shim changed the type to EfiBootServicesData the kernel bug was exposed # Fix: + Both EfiRuntimeServicesData and EfiBootServicesData must be mapped as + encrypted if SEV is active, as per: + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8d651ee9c71bb12fc0c8eb2786b66cbe5aa3e43b - # Regression potential The fix could potentially cause boot failures, if a memory region is marked encrypted when it isn't. I assume it would cause a panic similar to the one seen here for this bug: general protection fault, probably for non-canonical address 0x314836c31124d346: 0000 [#1] SMP NOPTI ** Description changed: # Overview Hirsute and Impish daily builds are currently not booting on Google Confidential Compute. Confidential compute is Google's platform that enables the use of Secure Encrypted Virtualization extension via AMD EPYC CPUs. Booting an image with version 1.45 works, but once upgraded to 1.47, the VM no longer boots, and instead the kernel panics. Launching the image with secure boot, but without confidential compute works as expected. # Expected result The system is able to reboot after the upgrade. # Actual result Kernel panic: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Steps to reproduce Launch a VM in GCE with confidential compute enabled with a serial v20210511a or later and look at the serial log for the kernel panic. Example CLI command to launch a VM: $ gcloud beta compute instances create $USER-confidential-testing --zone =us-west1-b --machine-type=n2d-standard-2 --image=daily-ubuntu-2104 -hirsute-v20210511a --image-project=ubuntu-os-cloud-devel --confidential-compute --maintenance-policy=TERMINATE The last known good working image is daily- ubuntu-2104-hirsute-v20210510. The upgrade that fails is when shim signed is updated from 1.46+15.4-0ubuntu1 to 1.47+15.4-0ubuntu2 # Logs & notes * 20210510 manifest (good): https://paste.ubuntu.com/p/QjnMPcJj7G/ * 20210511a manifest (bad): https://paste.ubuntu.com/p/PvJQwRXHcG/ * diff between manifests: https://paste.ubuntu.com/p/4nJtGxqGn7/ * serial logs of failed boot: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Cause: shim changed the memory type for pages reserved for EFI runtime services, from EfiRuntimeServicesData to EfiBootServicesData. Memory reserved for EFI runtime/boot services must be remapped as encrypted in the kernel (during boot) if SEV (secure encrypted virtualization) is enabled. The original kernel implementation of ioremap only correctly mapped the region as encrypted for EfiRuntimeServicesData regions, so when shim changed the type to EfiBootServicesData the kernel bug was exposed # Fix: Both EfiRuntimeServicesData and EfiBootServicesData must be mapped as encrypted if SEV is active, as per: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8d651ee9c71bb12fc0c8eb2786b66cbe5aa3e43b # Regression potential The fix could potentially cause boot failures, if a memory region is - marked encrypted when it isn't. I assume it would cause a panic similar - to the one seen here for this bug: + marked encrypted when it shouldn't be. I assume in that case it would + cause a panic similar to the one seen here for this bug: general protection fault, probably for non-canonical address 0x314836c31124d346: 0000 [#1] SMP NOPTI ** Also affects: linux (Ubuntu) Importance: Undecided Status: New ** No longer affects: linux-gcp (Ubuntu) ** No longer affects: linux-gcp (Ubuntu Hirsute) ** Changed in: linux (Ubuntu) Assignee: (unassigned) => Khaled El Mously (kmously) ** Changed in: linux (Ubuntu Hirsute) Assignee: (unassigned) => Khaled El Mously (kmously) ** Changed in: linux (Ubuntu) Status: New => In Progress ** Changed in: linux (Ubuntu Hirsute) Status: New => In Progress ** Description changed: # Overview Hirsute and Impish daily builds are currently not booting on Google Confidential Compute. Confidential compute is Google's platform that enables the use of Secure Encrypted Virtualization extension via AMD EPYC CPUs. Booting an image with version 1.45 works, but once upgraded to 1.47, the VM no longer boots, and instead the kernel panics. Launching the image with secure boot, but without confidential compute works as expected. # Expected result The system is able to reboot after the upgrade. # Actual result Kernel panic: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Steps to reproduce Launch a VM in GCE with confidential compute enabled with a serial v20210511a or later and look at the serial log for the kernel panic. Example CLI command to launch a VM: $ gcloud beta compute instances create $USER-confidential-testing --zone =us-west1-b --machine-type=n2d-standard-2 --image=daily-ubuntu-2104 -hirsute-v20210511a --image-project=ubuntu-os-cloud-devel --confidential-compute --maintenance-policy=TERMINATE The last known good working image is daily- ubuntu-2104-hirsute-v20210510. The upgrade that fails is when shim signed is updated from 1.46+15.4-0ubuntu1 to 1.47+15.4-0ubuntu2 # Logs & notes * 20210510 manifest (good): https://paste.ubuntu.com/p/QjnMPcJj7G/ * 20210511a manifest (bad): https://paste.ubuntu.com/p/PvJQwRXHcG/ * diff between manifests: https://paste.ubuntu.com/p/4nJtGxqGn7/ * serial logs of failed boot: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Cause: shim changed the memory type for pages reserved for EFI runtime services, from EfiRuntimeServicesData to EfiBootServicesData. Memory reserved for EFI runtime/boot services must be remapped as encrypted in the kernel (during boot) if SEV (secure encrypted virtualization) is enabled. The original kernel implementation of ioremap only correctly mapped the region as encrypted for EfiRuntimeServicesData regions, so when shim changed the type to EfiBootServicesData the kernel bug was exposed + Note that this affects all 5.11 kernels not just gcp + # Fix: Both EfiRuntimeServicesData and EfiBootServicesData must be mapped as encrypted if SEV is active, as per: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8d651ee9c71bb12fc0c8eb2786b66cbe5aa3e43b # Regression potential The fix could potentially cause boot failures, if a memory region is marked encrypted when it shouldn't be. I assume in that case it would cause a panic similar to the one seen here for this bug: general protection fault, probably for non-canonical address 0x314836c31124d346: 0000 [#1] SMP NOPTI ** Description changed: # Overview Hirsute and Impish daily builds are currently not booting on Google Confidential Compute. Confidential compute is Google's platform that enables the use of Secure Encrypted Virtualization extension via AMD EPYC CPUs. Booting an image with version 1.45 works, but once upgraded to 1.47, the VM no longer boots, and instead the kernel panics. Launching the image with secure boot, but without confidential compute works as expected. # Expected result The system is able to reboot after the upgrade. # Actual result Kernel panic: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Steps to reproduce Launch a VM in GCE with confidential compute enabled with a serial v20210511a or later and look at the serial log for the kernel panic. Example CLI command to launch a VM: $ gcloud beta compute instances create $USER-confidential-testing --zone =us-west1-b --machine-type=n2d-standard-2 --image=daily-ubuntu-2104 -hirsute-v20210511a --image-project=ubuntu-os-cloud-devel --confidential-compute --maintenance-policy=TERMINATE The last known good working image is daily- ubuntu-2104-hirsute-v20210510. The upgrade that fails is when shim signed is updated from 1.46+15.4-0ubuntu1 to 1.47+15.4-0ubuntu2 # Logs & notes * 20210510 manifest (good): https://paste.ubuntu.com/p/QjnMPcJj7G/ * 20210511a manifest (bad): https://paste.ubuntu.com/p/PvJQwRXHcG/ * diff between manifests: https://paste.ubuntu.com/p/4nJtGxqGn7/ * serial logs of failed boot: https://paste.ubuntu.com/p/mHrvVc6qBc/ # Cause: shim changed the memory type for pages reserved for EFI runtime services, from EfiRuntimeServicesData to EfiBootServicesData. Memory reserved for EFI runtime/boot services must be remapped as encrypted in the kernel (during boot) if SEV (secure encrypted virtualization) is enabled. The original kernel implementation of ioremap only correctly mapped the region as encrypted for EfiRuntimeServicesData regions, so when shim changed the type to EfiBootServicesData the kernel bug was exposed - Note that this affects all 5.11 kernels not just gcp + Note that this affects all 5.11 kernels not just gcp. It is possible + that gcp is the only cloud that uses sev currently (for "Confidential + Computing"). # Fix: Both EfiRuntimeServicesData and EfiBootServicesData must be mapped as encrypted if SEV is active, as per: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8d651ee9c71bb12fc0c8eb2786b66cbe5aa3e43b # Regression potential The fix could potentially cause boot failures, if a memory region is marked encrypted when it shouldn't be. I assume in that case it would cause a panic similar to the one seen here for this bug: general protection fault, probably for non-canonical address 0x314836c31124d346: 0000 [#1] SMP NOPTI -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1931254 Title: Google Confidential Compute fails to boot with shim version 1.47 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1931254/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
