I have a bit of an update on the sev-snp delay in booting. I am still trying to put everything together, and I will finalise everything on Monday after I finish going through all of the logs I generated today.
I traced through apic/x2apic call stack and I've been digging through the kernel code, and I suspect I know the issue. I think it comes down to a mixture of SNP-SEV support not being fully available in the kernel, and in the hypervisor with how it handles the APIC (the interrupt controller). Commit 1dfe571c12cf introduced the kernel support for kvm handling of SEV-SNP. This was not added to the kernel until kernel version 6.11. I need to dig through the kvm source to see how it handles the apix/x2apic with SNP-SEV still. Without the support in the kernel and hypervisor, it seems that the behaviour is somewhat undefined and relies on what is happening in the other guest VMs on the machine. This will explain why the pause does not hit every time you boot, but once it does hit, it reproduces every time.  One thing I have noticed is that when a VM f first started, it does not always happen. After a several reboots it usually does happen. Once it has happened once though, it happens every reboot after that.  I noticed GCP supports SEV-SNP, so I created an instance there, and installed the AWS kernel. I then installed the GCP kernel on the AWS instance. The AWS instance is hitting the pause with the GCP, and the AWS kernel, while the GCP instance has not hit it after about 100 reboots using the AWS kernel.. This makes me think it is the hypervisor rather than the kernel causing the issue. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2076217 Title: booting an 24.10 or 24.04 ec2 instance with SEV-SNP enabled hangs sometimes To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/2076217/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
