On 2/6/20 3:35 AM, Laszlo Ersek wrote: > On 02/05/20 00:01, Lendacky, Thomas wrote: >> BZ: >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2198&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cf4521a4f24d54b8f122108d7aae7de74%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637165785175188824&sdata=l7ff9Q7OFJ5JYGHt0meU04nBuM9aP1o5RT0fl8s9PoA%3D&reserved=0 >> >> Typically, an AP is booted using the INIT-SIPI-SIPI sequence. This >> sequence is intercepted by the hypervisor, which sets the AP's registers >> to the values requested by the sequence. At that point, the hypervisor can >> start the AP, which will then begin execution at the appropriate location. >> >> Under SEV-ES, AP booting presents some challenges since the hypervisor is >> not allowed to alter the AP's register state. In this situation, we have >> to distinguish between the AP's first boot and AP's subsequent boots. >> >> First boot: >> Once the AP's register state has been defined (which is before the guest >> is first booted) it cannot be altered. Should the hypervisor attempt to >> alter the register state, the change would be detected by the hardware >> and the VMRUN instruction would fail. Given this, the first boot for the >> AP is required to begin execution with this initial register state, which >> is typically the reset vector. This prevents the BSP from directing the >> AP startup location through the INIT-SIPI-SIPI sequence. >> >> To work around this, the firmware will provide a build time reserved area >> that can be used as the initial IP value. The hypervisor can extract this >> location value by checking for the SEV-ES reset block GUID that must be >> located 48-bytes from the end of the firmware. The format of the SEV-ES >> reset block area is: >> >> 0x00 - 0x01 - SEV-ES Reset IP >> 0x02 - 0x03 - SEV-ES Reset CS Segment Base[32:16] > > Slight typo here I think: in the next patch, you correctly write [31:16].
Yup, good catch, should be 31. Thanks, Tom > > Thanks > Laszlo > >> 0x04 - 0x05 - Size of the SEV-ES reset block >> 0x06 - 0x15 - SEV-ES Reset Block GUID >> (00f771de-1a7e-4fcb-890e-68c77e2fb44e) >> >> The total size is 22 bytes. Any expansion to this block must be done >> by adding new values before existing values. >> >> The hypervisor will use the IP and CS values obtained from the SEV-ES >> reset block to set as the AP's initial values. The CS Segment Base >> represents the upper 16 bits of the CS segment base and must be left >> shifted by 16 bits to form the complete CS segment base value. >> >> Before booting the AP for the first time, the BSP must initialize the >> SEV-ES reset area. This consists of programming a FAR JMP instruction >> to the contents of a memory location that is also located in the SEV-ES >> reset area. The BSP must program the IP and CS values for the FAR JMP >> based on values drived from the INIT-SIPI-SIPI sequence. >> >> Subsequent boots: >> Again, the hypervisor cannot alter the AP register state, so a method is >> required to take the AP out of halt state and redirect it to the desired >> IP location. If it is determined that the AP is running in an SEV-ES >> guest, then instead of calling CpuSleep(), a VMGEXIT is issued with the >> AP Reset Hold exit code (0x80000004). The hypervisor will put the AP in >> a halt state, waiting for an INIT-SIPI-SIPI sequence. Once the sequence >> is recognized, the hypervisor will resume the AP. At this point the AP >> must transition from the current 64-bit long mode down to 16-bit real >> mode and begin executing at the derived location from the INIT-SIPI-SIPI >> sequence. >> >> Another change is around the area of obtaining the (x2)APIC ID during AP >> startup. During AP startup, the AP can't take a #VC exception before the >> AP has established a stack. However, the AP stack is set by using the >> (x2)APIC ID, which is obtained through CPUID instructions. A CPUID >> instruction will cause a #VC, so a different method must be used. The >> GHCB protocol supports a method to obtain CPUID information from the >> hypervisor through the GHCB MSR. This method does not require a stack, >> so it is used to obtain the necessary CPUID information to determine the >> (x2)APIC ID. >> >> The new 16-bit protected mode GDT entry is used in order to transition >> from 64-bit long mode down to 16-bit real mode. >> >> A new assembler routine is created that takes the AP from 64-bit long mode >> to 16-bit real mode. This is located under 1MB in memory and transitions >> from 64-bit long mode to 32-bit compatibility mode to 16-bit protected >> mode and finally 16-bit real mode. >> >> Cc: Eric Dong <eric.d...@intel.com> >> Cc: Ray Ni <ray...@intel.com> >> Cc: Laszlo Ersek <ler...@redhat.com> >> Signed-off-by: Tom Lendacky <thomas.lenda...@amd.com> >> --- >> UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 3 + >> UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 3 + >> UefiCpuPkg/Library/MpInitLib/MpLib.h | 60 ++++ >> UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 70 ++++- >> UefiCpuPkg/Library/MpInitLib/MpLib.c | 256 +++++++++++++++++- >> UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 19 ++ >> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c | 2 +- >> UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 2 +- >> .../Library/MpInitLib/Ia32/MpFuncs.nasm | 15 + >> UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 4 +- >> UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 239 ++++++++++++++++ >> 11 files changed, 659 insertions(+), 14 deletions(-) >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> index 2c26f20c1972..c52951651851 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf >> @@ -51,6 +51,7 @@ [LibraryClasses] >> UefiBootServicesTableLib >> DebugAgentLib >> SynchronizationLib >> + VmgExitLib >> >> [Protocols] >> gEfiTimerArchProtocolGuid ## SOMETIMES_CONSUMES >> @@ -70,5 +71,7 @@ [Pcd] >> gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## >> SOMETIMES_CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdCpuShadowMicrocodeByFit ## >> CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled ## >> CONSUMES >> + gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase ## >> SOMETIMES_CONSUMES >> gEfiMdeModulePkgTokenSpaceGuid.PcdCpuStackGuard ## >> CONSUMES >> + gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase ## >> CONSUMES >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> index 66b2acfe98e7..ff392aeec763 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf >> @@ -50,6 +50,7 @@ [LibraryClasses] >> UefiCpuLib >> SynchronizationLib >> PeiServicesLib >> + VmgExitLib >> >> [Pcd] >> gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber ## >> CONSUMES >> @@ -62,6 +63,8 @@ [Pcd] >> gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## >> SOMETIMES_CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdCpuShadowMicrocodeByFit ## >> CONSUMES >> gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled ## >> CONSUMES >> + gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase ## >> SOMETIMES_CONSUMES >> + gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase ## >> CONSUMES >> >> [Guids] >> gEdkiiS3SmmInitDoneGuid >> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h >> b/UefiCpuPkg/Library/MpInitLib/MpLib.h >> index 864b16872010..63d81ac3e42e 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h >> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h >> @@ -170,6 +170,11 @@ typedef struct { >> UINT8 *RelocateApLoopFuncAddress; >> UINTN RelocateApLoopFuncSize; >> UINTN ModeTransitionOffset; >> + UINTN SwitchToRealSize; >> + UINTN SwitchToRealOffset; >> + UINTN SwitchToRealNoNxOffset; >> + UINTN SwitchToRealPM16ModeOffset; >> + UINTN SwitchToRealPM16ModeSize; >> } MP_ASSEMBLY_ADDRESS_MAP; >> >> typedef struct _CPU_MP_DATA CPU_MP_DATA; >> @@ -208,6 +213,8 @@ typedef struct { >> // Enable5LevelPaging indicates whether 5-level paging is enabled in long >> mode. >> // >> BOOLEAN Enable5LevelPaging; >> + BOOLEAN SevEsIsEnabled; >> + UINTN GhcbBase; >> } MP_CPU_EXCHANGE_INFO; >> >> #pragma pack() >> @@ -256,6 +263,7 @@ struct _CPU_MP_DATA { >> UINT8 ApLoopMode; >> UINT8 ApTargetCState; >> UINT16 PmCodeSegment; >> + UINT16 Pm16CodeSegment; >> CPU_AP_DATA *CpuData; >> volatile MP_CPU_EXCHANGE_INFO *MpCpuExchangeInfo; >> >> @@ -275,8 +283,47 @@ struct _CPU_MP_DATA { >> BOOLEAN WakeUpByInitSipiSipi; >> >> BOOLEAN SevEsIsEnabled; >> + UINTN SevEsAPBuffer; >> + UINTN SevEsAPResetStackStart; >> + CPU_MP_DATA *NewCpuMpData; >> + >> + UINT64 GhcbBase; >> }; >> >> +#define AP_RESET_STACK_SIZE 64 >> + >> +#pragma pack(1) >> + >> +typedef struct { >> + UINT8 InsnBuffer[8]; >> + UINT16 Rip; >> + UINT16 Segment; >> +} SEV_ES_AP_JMP_FAR; >> + >> +#pragma pack() >> + >> +/** >> + Assembly code to move an AP from long mode to real mode. >> + >> + Move an AP from long mode to real mode in preparation to invoking >> + the reset vector. This is used for SEV-ES guests where a hypervisor >> + is not allowed to set the CS and RIP to point to the reset vector. >> + >> + @param[in] BufferStart The reset vector target. >> + @param[in] Code16 16-bit protected mode code segment value. >> + @param[in] Code32 32-bit protected mode code segment value. >> + @param[in] StackStart The start of a stack to be used for transitioning >> + from long mode to real mode. >> +**/ >> +typedef >> +VOID >> +(EFIAPI AP_RESET) ( >> + IN UINTN BufferStart, >> + IN UINT16 Code16, >> + IN UINT16 Code32, >> + IN UINTN StackStart >> + ); >> + >> extern EFI_GUID mCpuInitMpLibHobGuid; >> >> /** >> @@ -382,6 +429,19 @@ GetModeTransitionBuffer ( >> IN UINTN BufferSize >> ); >> >> +/** >> + Return the address of the SEV-ES AP jump table. >> + >> + This buffer is required in order for an SEV-ES guest to transition from >> + UEFI into an OS. >> + >> + @retval other Return SEV-ES AP jump table buffer >> +**/ >> +UINTN >> +GetSevEsAPMemory ( >> + VOID >> + ); >> + >> /** >> This function will be called by BSP to wakeup AP. >> >> diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> index b17e287bbf49..8df5b6d919e6 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c >> @@ -12,6 +12,8 @@ >> #include <Library/UefiBootServicesTableLib.h> >> #include <Library/DebugAgentLib.h> >> #include <Library/DxeServicesTableLib.h> >> +#include <Register/Amd/Fam17Msr.h> >> +#include <Register/Amd/Ghcb.h> >> >> #include <Protocol/Timer.h> >> >> @@ -145,6 +147,39 @@ GetModeTransitionBuffer ( >> return (UINTN)StartAddress; >> } >> >> +/** >> + Return the address of the SEV-ES AP jump table. >> + >> + This buffer is required in order for an SEV-ES guest to transition from >> + UEFI into an OS. >> + >> + @retval other Return SEV-ES AP jump table buffer >> +**/ >> +UINTN >> +GetSevEsAPMemory ( >> + VOID >> + ) >> +{ >> + EFI_STATUS Status; >> + EFI_PHYSICAL_ADDRESS StartAddress; >> + >> + // >> + // Allocate 1 page for AP jump table page >> + // >> + StartAddress = BASE_4GB - 1; >> + Status = gBS->AllocatePages ( >> + AllocateMaxAddress, >> + EfiReservedMemoryType, >> + 1, >> + &StartAddress >> + ); >> + ASSERT_EFI_ERROR (Status); >> + >> + DEBUG ((DEBUG_INFO, "Dxe: SevEsAPMemory = %lx\n", (UINTN) StartAddress)); >> + >> + return (UINTN) StartAddress; >> +} >> + >> /** >> Checks APs status and updates APs status if needed. >> >> @@ -219,6 +254,38 @@ CheckApsStatus ( >> } >> } >> >> +/** >> + Get Protected mode code segment with 16-bit default addressing >> + from current GDT table. >> + >> + @return Protected mode 16-bit code segment value. >> +**/ >> +UINT16 >> +GetProtectedMode16CS ( >> + VOID >> + ) >> +{ >> + IA32_DESCRIPTOR GdtrDesc; >> + IA32_SEGMENT_DESCRIPTOR *GdtEntry; >> + UINTN GdtEntryCount; >> + UINT16 Index; >> + >> + Index = (UINT16) -1; >> + AsmReadGdtr (&GdtrDesc); >> + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); >> + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> + for (Index = 0; Index < GdtEntryCount; Index++) { >> + if (GdtEntry->Bits.L == 0) { >> + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 0) { >> + break; >> + } >> + } >> + GdtEntry++; >> + } >> + ASSERT (Index != GdtEntryCount); >> + return Index * 8; >> +} >> + >> /** >> Get Protected mode code segment from current GDT table. >> >> @@ -239,7 +306,7 @@ GetProtectedModeCS ( >> GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> for (Index = 0; Index < GdtEntryCount; Index++) { >> if (GdtEntry->Bits.L == 0) { >> - if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) { >> + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) { >> break; >> } >> } >> @@ -301,6 +368,7 @@ MpInitChangeApLoopCallback ( >> >> CpuMpData = GetCpuMpData (); >> CpuMpData->PmCodeSegment = GetProtectedModeCS (); >> + CpuMpData->Pm16CodeSegment = GetProtectedMode16CS (); >> CpuMpData->ApLoopMode = PcdGet8 (PcdCpuApLoopMode); >> mNumberToFinish = CpuMpData->CpuCount - 1; >> WakeUpAP (CpuMpData, TRUE, 0, RelocateApLoop, NULL, TRUE); >> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c >> b/UefiCpuPkg/Library/MpInitLib/MpLib.c >> index 5e3183c2493b..ca8a3a3a7be9 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c >> @@ -7,6 +7,9 @@ >> **/ >> >> #include "MpLib.h" >> +#include <Library/VmgExitLib.h> >> +#include <Register/Amd/Fam17Msr.h> >> +#include <Register/Amd/Ghcb.h> >> >> EFI_GUID mCpuInitMpLibHobGuid = CPU_INIT_MP_LIB_HOB_GUID; >> >> @@ -288,6 +291,14 @@ GetApLoopMode ( >> // >> ApLoopMode = ApInHltLoop; >> } >> + >> + if (PcdGetBool (PcdSevEsIsEnabled)) { >> + // >> + // For SEV-ES, force AP in Hlt-loop mode in order to use the GHCB >> + // protocol for starting APs >> + // >> + ApLoopMode = ApInHltLoop; >> + } >> } >> >> if (ApLoopMode != ApInMwaitLoop) { >> @@ -579,6 +590,108 @@ InitializeApData ( >> SetApState (&CpuMpData->CpuData[ProcessorNumber], CpuStateIdle); >> } >> >> +/** >> + Get Protected mode code segment with 16-bit default addressing >> + from current GDT table. >> + >> + @return Protected mode 16-bit code segment value. >> +**/ >> +STATIC >> +UINT16 >> +GetProtectedMode16CS ( >> + VOID >> + ) >> +{ >> + IA32_DESCRIPTOR GdtrDesc; >> + IA32_SEGMENT_DESCRIPTOR *GdtEntry; >> + UINTN GdtEntryCount; >> + UINT16 Index; >> + >> + Index = (UINT16) -1; >> + AsmReadGdtr (&GdtrDesc); >> + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); >> + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> + for (Index = 0; Index < GdtEntryCount; Index++) { >> + if (GdtEntry->Bits.L == 0 && >> + GdtEntry->Bits.DB == 0 && >> + GdtEntry->Bits.Type > 8) { >> + break; >> + } >> + GdtEntry++; >> + } >> + ASSERT (Index != GdtEntryCount); >> + return Index * 8; >> +} >> + >> +/** >> + Get Protected mode code segment with 32-bit default addressing >> + from current GDT table. >> + >> + @return Protected mode 32-bit code segment value. >> +**/ >> +STATIC >> +UINT16 >> +GetProtectedMode32CS ( >> + VOID >> + ) >> +{ >> + IA32_DESCRIPTOR GdtrDesc; >> + IA32_SEGMENT_DESCRIPTOR *GdtEntry; >> + UINTN GdtEntryCount; >> + UINT16 Index; >> + >> + Index = (UINT16) -1; >> + AsmReadGdtr (&GdtrDesc); >> + GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR); >> + GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> + for (Index = 0; Index < GdtEntryCount; Index++) { >> + if (GdtEntry->Bits.L == 0 && >> + GdtEntry->Bits.DB == 1 && >> + GdtEntry->Bits.Type > 8) { >> + break; >> + } >> + GdtEntry++; >> + } >> + ASSERT (Index != GdtEntryCount); >> + return Index * 8; >> +} >> + >> +/** >> + Reset an AP when in SEV-ES mode. >> + >> + @retval EFI_DEVICE_ERROR Reset of AP failed. >> +**/ >> +STATIC >> +VOID >> +MpInitLibSevEsAPReset ( >> + GHCB *Ghcb, >> + CPU_MP_DATA *CpuMpData >> + ) >> +{ >> + UINT16 Code16, Code32; >> + AP_RESET *APResetFn; >> + UINTN BufferStart; >> + UINTN StackStart; >> + >> + Code16 = GetProtectedMode16CS (); >> + Code32 = GetProtectedMode32CS (); >> + >> + if (CpuMpData->WakeupBufferHigh != 0) { >> + APResetFn = (AP_RESET *) (CpuMpData->WakeupBufferHigh + >> CpuMpData->AddressMap.SwitchToRealNoNxOffset); >> + } else { >> + APResetFn = (AP_RESET *) (CpuMpData->MpCpuExchangeInfo->BufferStart + >> CpuMpData->AddressMap.SwitchToRealOffset); >> + } >> + >> + BufferStart = CpuMpData->MpCpuExchangeInfo->BufferStart; >> + StackStart = CpuMpData->SevEsAPResetStackStart - >> + (AP_RESET_STACK_SIZE * GetApicId ()); >> + >> + // >> + // This call never returns. >> + // >> + APResetFn (BufferStart, Code16, Code32, StackStart); >> +} >> + >> /** >> This function will be called from AP reset code if BSP uses WakeUpAP. >> >> @@ -734,7 +847,28 @@ ApWakeupFunction ( >> // >> while (TRUE) { >> DisableInterrupts (); >> - CpuSleep (); >> + if (CpuMpData->SevEsIsEnabled) { >> + MSR_SEV_ES_GHCB_REGISTER Msr; >> + GHCB *Ghcb; >> + >> + Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB); >> + Ghcb = Msr.Ghcb; >> + >> + VmgInit (Ghcb); >> + VmgExit (Ghcb, SvmExitApResetHold, 0, 0); >> + /*TODO: Check return value to verify SIPI issued */ >> + >> + // >> + // Awakened in a new phase? Use the new CpuMpData >> + // >> + if (CpuMpData->NewCpuMpData) { >> + CpuMpData = CpuMpData->NewCpuMpData; >> + } >> + >> + MpInitLibSevEsAPReset (Ghcb, CpuMpData); >> + } else { >> + CpuSleep (); >> + } >> CpuPause (); >> } >> } >> @@ -847,6 +981,9 @@ FillExchangeInfoData ( >> ExchangeInfo->Enable5LevelPaging = (BOOLEAN) (Cr4.Bits.LA57 == 1); >> DEBUG ((DEBUG_INFO, "%a: 5-Level Paging = %d\n", gEfiCallerBaseName, >> ExchangeInfo->Enable5LevelPaging)); >> >> + ExchangeInfo->SevEsIsEnabled = CpuMpData->SevEsIsEnabled; >> + ExchangeInfo->GhcbBase = CpuMpData->GhcbBase; >> + >> // >> // Get the BSP's data of GDT and IDT >> // >> @@ -873,8 +1010,9 @@ FillExchangeInfoData ( >> // EfiBootServicesCode to avoid page fault if NX memory protection is >> enabled. >> // >> if (CpuMpData->WakeupBufferHigh != 0) { >> - Size = CpuMpData->AddressMap.RendezvousFunnelSize - >> - CpuMpData->AddressMap.ModeTransitionOffset; >> + Size = CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize - >> + CpuMpData->AddressMap.ModeTransitionOffset; >> CopyMem ( >> (VOID *)CpuMpData->WakeupBufferHigh, >> CpuMpData->AddressMap.RendezvousFunnelAddress + >> @@ -927,7 +1065,8 @@ BackupAndPrepareWakeupBuffer( >> CopyMem ( >> (VOID *) CpuMpData->WakeupBuffer, >> (VOID *) CpuMpData->AddressMap.RendezvousFunnelAddress, >> - CpuMpData->AddressMap.RendezvousFunnelSize >> + CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize >> ); >> } >> >> @@ -948,6 +1087,40 @@ RestoreWakeupBuffer( >> ); >> } >> >> +/** >> + Calculate the size of the reset stack. >> +**/ >> +STATIC >> +UINTN >> +GetApResetStackSize( >> + VOID >> + ) >> +{ >> + return AP_RESET_STACK_SIZE * PcdGet32(PcdCpuMaxLogicalProcessorNumber); >> +} >> + >> +/** >> + Calculate the size of the reset vector. >> + >> + @param[in] AddressMap The pointer to Address Map structure. >> +**/ >> +STATIC >> +UINTN >> +GetApResetVectorSize( >> + IN MP_ASSEMBLY_ADDRESS_MAP *AddressMap >> + ) >> +{ >> + UINTN Size; >> + >> + Size = ALIGN_VALUE (AddressMap->RendezvousFunnelSize + >> + AddressMap->SwitchToRealSize + >> + sizeof (MP_CPU_EXCHANGE_INFO), >> + CPU_STACK_ALIGNMENT); >> + Size += GetApResetStackSize (); >> + >> + return Size; >> +} >> + >> /** >> Allocate reset vector buffer. >> >> @@ -961,16 +1134,22 @@ AllocateResetVector ( >> UINTN ApResetVectorSize; >> >> if (CpuMpData->WakeupBuffer == (UINTN) -1) { >> - ApResetVectorSize = CpuMpData->AddressMap.RendezvousFunnelSize + >> - sizeof (MP_CPU_EXCHANGE_INFO); >> + ApResetVectorSize = GetApResetVectorSize (&CpuMpData->AddressMap); >> >> CpuMpData->WakeupBuffer = GetWakeupBuffer (ApResetVectorSize); >> CpuMpData->MpCpuExchangeInfo = (MP_CPU_EXCHANGE_INFO *) (UINTN) >> - (CpuMpData->WakeupBuffer + >> CpuMpData->AddressMap.RendezvousFunnelSize); >> + (CpuMpData->WakeupBuffer + >> + CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize); >> CpuMpData->WakeupBufferHigh = GetModeTransitionBuffer ( >> - >> CpuMpData->AddressMap.RendezvousFunnelSize - >> + >> CpuMpData->AddressMap.RendezvousFunnelSize + >> + CpuMpData->AddressMap.SwitchToRealSize - >> >> CpuMpData->AddressMap.ModeTransitionOffset >> ); >> + // >> + // The reset stack starts at the end of the buffer. >> + // >> + CpuMpData->SevEsAPResetStackStart = CpuMpData->WakeupBuffer + >> ApResetVectorSize; >> } >> BackupAndPrepareWakeupBuffer (CpuMpData); >> } >> @@ -985,7 +1164,31 @@ FreeResetVector ( >> IN CPU_MP_DATA *CpuMpData >> ) >> { >> - RestoreWakeupBuffer (CpuMpData); >> + // >> + // If SEV-ES is enabled, the reset area is needed for AP parking and >> + // and AP startup in the OS, so the reset area is reserved. Do not >> + // perform the restore as this will overwrite memory which has data >> + // needed by SEV-ES. >> + // >> + if (!CpuMpData->SevEsIsEnabled) { >> + RestoreWakeupBuffer (CpuMpData); >> + } >> +} >> + >> +/** >> + Allocate the SEV-ES AP jump table buffer. >> + >> + @param[in, out] CpuMpData The pointer to CPU MP Data structure. >> +**/ >> +VOID >> +AllocateSevEsAPMemory ( >> + IN OUT CPU_MP_DATA *CpuMpData >> + ) >> +{ >> + if (CpuMpData->SevEsAPBuffer == (UINTN) -1) { >> + CpuMpData->SevEsAPBuffer = >> + CpuMpData->SevEsIsEnabled ? GetSevEsAPMemory () : 0; >> + } >> } >> >> /** >> @@ -1022,6 +1225,7 @@ WakeUpAP ( >> CpuMpData->InitFlag != ApInitDone) { >> ResetVectorRequired = TRUE; >> AllocateResetVector (CpuMpData); >> + AllocateSevEsAPMemory (CpuMpData); >> FillExchangeInfoData (CpuMpData); >> SaveLocalApicTimerSetting (CpuMpData); >> } >> @@ -1058,6 +1262,35 @@ WakeUpAP ( >> } >> } >> if (ResetVectorRequired) { >> + // >> + // For SEV-ES, the initial AP boot address will be defined by >> + // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address >> + // from the original INIT-SIPI-SIPI. >> + // >> + if (CpuMpData->SevEsIsEnabled) { >> + SEV_ES_AP_JMP_FAR *JmpFar; >> + UINT32 Offset, InsnByte; >> + UINT8 LoNib, HiNib; >> + >> + JmpFar = (SEV_ES_AP_JMP_FAR *) FixedPcdGet32 (PcdSevEsWorkAreaBase); >> + ASSERT (JmpFar != NULL); >> + >> + Offset = FixedPcdGet32 (PcdSevEsWorkAreaBase); >> + Offset += sizeof(JmpFar->InsnBuffer); >> + LoNib = (UINT8) Offset; >> + HiNib = (UINT8) (Offset >> 8); >> + >> + // JMP FAR [CS:XXYY] => 2E FF 2E YY XX >> + InsnByte = 0; >> + JmpFar->InsnBuffer[InsnByte++] = 0x2E; // CS override prefix >> + JmpFar->InsnBuffer[InsnByte++] = 0xFF; // JMP (FAR) >> + JmpFar->InsnBuffer[InsnByte++] = 0x2E; // ModRM (JMP memory >> location) >> + JmpFar->InsnBuffer[InsnByte++] = LoNib; // YY offset ... >> + JmpFar->InsnBuffer[InsnByte++] = HiNib; // XX offset ... >> + >> + JmpFar->Rip = 0; >> + JmpFar->Segment = (UINT16) (ExchangeInfo->BufferStart >> 4); >> + } >> // >> // Wakeup all APs >> // >> @@ -1625,7 +1858,7 @@ MpInitLibInitialize ( >> ASSERT (MaxLogicalProcessorNumber != 0); >> >> AsmGetAddressMap (&AddressMap); >> - ApResetVectorSize = AddressMap.RendezvousFunnelSize + sizeof >> (MP_CPU_EXCHANGE_INFO); >> + ApResetVectorSize = GetApResetVectorSize (&AddressMap); >> ApStackSize = PcdGet32(PcdCpuApStackSize); >> ApLoopMode = GetApLoopMode (&MonitorFilterSize); >> >> @@ -1688,6 +1921,8 @@ MpInitLibInitialize ( >> } >> InitializeSpinLock(&CpuMpData->MpLock); >> CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled); >> + CpuMpData->SevEsAPBuffer = (UINTN) -1; >> + CpuMpData->GhcbBase = PcdGet64 (PcdGhcbBase); >> >> // >> // Make sure no memory usage outside of the allocated buffer. >> @@ -1751,6 +1986,7 @@ MpInitLibInitialize ( >> // APs have been wakeup before, just get the CPU Information >> // from HOB >> // >> + OldCpuMpData->NewCpuMpData = CpuMpData; >> CpuMpData->CpuCount = OldCpuMpData->CpuCount; >> CpuMpData->BspNumber = OldCpuMpData->BspNumber; >> CpuMpData->CpuInfoInHob = OldCpuMpData->CpuInfoInHob; >> diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> index 06e3f5d0d3da..e8103a9ce094 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c >> @@ -280,6 +280,25 @@ GetModeTransitionBuffer ( >> return 0; >> } >> >> +/** >> + Return the address of the SEV-ES AP jump table. >> + >> + This buffer is required in order for an SEV-ES guest to transition from >> + UEFI into an OS. >> + >> + @retval other Return SEV-ES AP jump table buffer >> +**/ >> +UINTN >> +GetSevEsAPMemory ( >> + VOID >> + ) >> +{ >> + // >> + // PEI phase doesn't need to do such transition. So simply return 0. >> + // >> + return 0; >> +} >> + >> /** >> Checks APs status and updates APs status if needed. >> >> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> index 6298571e29b2..28f8e8e133e5 100644 >> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c >> @@ -121,7 +121,7 @@ GetProtectedModeCS ( >> GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base; >> for (Index = 0; Index < GdtEntryCount; Index++) { >> if (GdtEntry->Bits.L == 0) { >> - if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.L == 0) { >> + if (GdtEntry->Bits.Type > 8 && GdtEntry->Bits.DB == 1) { >> break; >> } >> } >> diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> index efb1bc2bf7cb..4f5a7c859a56 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc >> @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE equ 0 >> CPU_SWITCH_STATE_STORED equ 1 >> CPU_SWITCH_STATE_LOADED equ 2 >> >> -LockLocation equ (RendezvousFunnelProcEnd - >> RendezvousFunnelProcStart) >> +LockLocation equ (SwitchToRealProcEnd - >> RendezvousFunnelProcStart) >> StackStartAddressLocation equ LockLocation + 04h >> StackSizeLocation equ LockLocation + 08h >> ApProcedureLocation equ LockLocation + 0Ch >> diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> index b74046b76af3..309d53bf3b37 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> +++ b/UefiCpuPkg/Library/MpInitLib/Ia32/MpFuncs.nasm >> @@ -215,6 +215,16 @@ CProcedureInvoke: >> jmp $ ; Never reach here >> RendezvousFunnelProcEnd: >> >> +;------------------------------------------------------------------------------------- >> +;SwitchToRealProc procedure follows. >> +;NOT USED IN 32 BIT MODE. >> +;------------------------------------------------------------------------------------- >> +global ASM_PFX(SwitchToRealProc) >> +ASM_PFX(SwitchToRealProc): >> +SwitchToRealProcStart: >> + jmp $ ; Never reach here >> +SwitchToRealProcEnd: >> + >> >> ;------------------------------------------------------------------------------------- >> ; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, >> TopOfApStack, CountTofinish); >> >> ;------------------------------------------------------------------------------------- >> @@ -263,6 +273,11 @@ ASM_PFX(AsmGetAddressMap): >> mov dword [ebx + 0Ch], AsmRelocateApLoopStart >> mov dword [ebx + 10h], AsmRelocateApLoopEnd - >> AsmRelocateApLoopStart >> mov dword [ebx + 14h], Flat32Start - RendezvousFunnelProcStart >> + mov dword [ebx + 18h], SwitchToRealProcEnd - >> SwitchToRealProcStart ; SwitchToRealSize >> + mov dword [ebx + 1Ch], SwitchToRealProcStart - >> RendezvousFunnelProcStart ; SwitchToRealOffset >> + mov dword [ebx + 20h], SwitchToRealProcStart - Flat32Start >> ; SwitchToRealNoNxOffset >> + mov dword [ebx + 24h], 0 >> ; SwitchToRealPM16ModeOffset >> + mov dword [ebx + 28h], 0 >> ; SwitchToRealPM16ModeSize >> >> popad >> ret >> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> index 58ef369342a7..c92daaaffd6b 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc >> @@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE equ 0 >> CPU_SWITCH_STATE_STORED equ 1 >> CPU_SWITCH_STATE_LOADED equ 2 >> >> -LockLocation equ (RendezvousFunnelProcEnd - >> RendezvousFunnelProcStart) >> +LockLocation equ (SwitchToRealProcEnd - >> RendezvousFunnelProcStart) >> StackStartAddressLocation equ LockLocation + 08h >> StackSizeLocation equ LockLocation + 10h >> ApProcedureLocation equ LockLocation + 18h >> @@ -41,3 +41,5 @@ ModeTransitionSegmentLocation equ LockLocation + 98h >> ModeHighMemoryLocation equ LockLocation + 9Ah >> ModeHighSegmentLocation equ LockLocation + 9Eh >> Enable5LevelPagingLocation equ LockLocation + 0A0h >> +SevEsIsEnabledLocation equ LockLocation + 0A1h >> +GhcbBaseLocation equ LockLocation + 0A2h >> diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> index 87f2523e856f..6956b408d004 100644 >> --- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> +++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm >> @@ -184,9 +184,97 @@ Releaselock: >> add edi, StackStartAddressLocation >> add rax, qword [edi] >> mov rsp, rax >> + >> + lea edi, [esi + SevEsIsEnabledLocation] >> + cmp byte [edi], 1 ; SevEsIsEnabled >> + jne CProcedureInvoke >> + >> + ; >> + ; program GHCB >> + ; Each page after the GHCB is a per-CPU page, so the calculation >> programs >> + ; a GHCB to be every 8KB. >> + ; >> + mov eax, SIZE_4KB >> + shl eax, 1 ; EAX = SIZE_4K * 2 >> + mov ecx, ebx >> + mul ecx ; EAX = SIZE_4K * 2 * >> CpuNumber >> + mov edi, esi >> + add edi, GhcbBaseLocation >> + add rax, qword [edi] >> + mov rdx, rax >> + shr rdx, 32 >> + mov rcx, 0xc0010130 >> + wrmsr >> jmp CProcedureInvoke >> >> GetApicId: >> + lea edi, [esi + SevEsIsEnabledLocation] >> + cmp byte [edi], 1 ; SevEsIsEnabled >> + jne DoCpuid >> + >> + ; >> + ; Since we don't have a stack yet, we can't take a #VC >> + ; exception. Use the GHCB protocol to perform the CPUID >> + ; calls. >> + ; >> + mov rcx, 0xc0010130 >> + rdmsr >> + shl rdx, 32 >> + or rax, rdx >> + mov rdi, rax ; RDI now holds the original GHCB GPA >> + >> + mov rdx, 0 ; CPUID function 0 >> + mov rax, 0 ; RAX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + cmp edx, 0bh >> + jb NoX2ApicSevEs ; CPUID level below >> CPUID_EXTENDED_TOPOLOGY >> + >> + mov rdx, 0bh ; CPUID function 0x0b >> + mov rax, 040000000h ; RBX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + test edx, 0ffffh >> + jz NoX2ApicSevEs ; CPUID.0BH:EBX[15:0] is zero >> + >> + mov rdx, 0bh ; CPUID function 0x0b >> + mov rax, 0c0000000h ; RDX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + >> + ; Processor is x2APIC capable; 32-bit x2APIC ID is now in EDX >> + jmp RestoreGhcb >> + >> +NoX2ApicSevEs: >> + ; Processor is not x2APIC capable, so get 8-bit APIC ID >> + mov rdx, 1 ; CPUID function 1 >> + mov rax, 040000000h ; RBX register requested >> + or rax, 4 >> + wrmsr >> + rep vmmcall >> + rdmsr >> + shr edx, 24 >> + >> +RestoreGhcb: >> + mov rbx, rdx ; Save x2APIC/APIC ID >> + >> + mov rdx, rdi ; RDI holds the saved GHCB GPA >> + shr rdx, 32 >> + mov eax, edi >> + wrmsr >> + >> + mov rdx, rbx >> + >> + ; x2APIC ID or APIC ID is in EDX >> + jmp GetProcessorNumber >> + >> +DoCpuid: >> mov eax, 0 >> cpuid >> cmp eax, 0bh >> @@ -253,12 +341,158 @@ CProcedureInvoke: >> >> RendezvousFunnelProcEnd: >> >> +;------------------------------------------------------------------------------------- >> +;SwitchToRealProc procedure follows. >> +;ALSO THIS PROCEDURE IS EXECUTED BY APs TRANSITIONING TO 16 BIT MODE. HENCE >> THIS PROC >> +;IS IN MACHINE CODE. >> +; SwitchToRealProc (UINTN BufferStart, UINT16 Code16, UINT16 Code32, UINTN >> StackStart) >> +; rcx - Buffer Start >> +; rdx - Code16 Selector Offset >> +; r8 - Code32 Selector Offset >> +; r9 - Stack Start >> +;------------------------------------------------------------------------------------- >> +global ASM_PFX(SwitchToRealProc) >> +ASM_PFX(SwitchToRealProc): >> +SwitchToRealProcStart: >> +BITS 64 >> + cli >> + >> + ; >> + ; Get RDX reset value before changing stacks since the >> + ; new stack won't be able to accomodate a #VC exception. >> + ; >> + push rax >> + push rbx >> + push rcx >> + push rdx >> + >> + mov rax, 1 >> + cpuid >> + mov rsi, rax ; Save off the reset value for >> RDX >> + >> + pop rdx >> + pop rcx >> + pop rbx >> + pop rax >> + >> + ; >> + ; Establish stack below 1MB >> + ; >> + mov rsp, r9 >> + >> + ; >> + ; Push ultimate Reset Vector onto the stack >> + ; >> + mov rax, rcx >> + shr rax, 4 >> + push word 0x0002 ; RFLAGS >> + push ax ; CS >> + push word 0x0000 ; RIP >> + push word 0x0000 ; For alignment, will be >> discarded >> + >> + ; >> + ; Get address of "16-bit operand size" label >> + ; >> + lea rbx, [PM16Mode] >> + >> + ; >> + ; Push addresses used to change to compatibility mode >> + ; >> + lea rax, [CompatMode] >> + push r8 >> + push rax >> + >> + ; >> + ; Clear R8 - R15, for reset, before going into 32-bit mode >> + ; >> + xor r8, r8 >> + xor r9, r9 >> + xor r10, r10 >> + xor r11, r11 >> + xor r12, r12 >> + xor r13, r13 >> + xor r14, r14 >> + xor r15, r15 >> + >> + ; >> + ; Far return into 32-bit mode >> + ; >> +o64 retf >> + >> +BITS 32 >> +CompatMode: >> + ; >> + ; Set up stack to prepare for exiting protected mode >> + ; >> + push edx ; Code16 CS >> + push ebx ; PM16Mode label address >> + >> + ; >> + ; Disable paging >> + ; >> + mov eax, cr0 ; Read CR0 >> + btr eax, 31 ; Set PG=0 >> + mov cr0, eax ; Write CR0 >> + >> + ; >> + ; Disable long mode >> + ; >> + mov ecx, 0c0000080h ; EFER MSR number >> + rdmsr ; Read EFER >> + btr eax, 8 ; Set LME=0 >> + wrmsr ; Write EFER >> + >> + ; >> + ; Disable PAE >> + ; >> + mov eax, cr4 ; Read CR4 >> + btr eax, 5 ; Set PAE=0 >> + mov cr4, eax ; Write CR4 >> + >> + mov edx, esi ; Restore RDX reset value >> + >> + ; >> + ; Switch to 16-bit operand size >> + ; >> + retf >> + >> +BITS 16 >> + ; >> + ; At entry to this label >> + ; - RDX will have its reset value >> + ; - On the top of the stack >> + ; - Alignment data (two bytes) to be discarded >> + ; - IP for Real Mode (two bytes) >> + ; - CS for Real Mode (two bytes) >> + ; >> +PM16Mode: >> + mov eax, cr0 ; Read CR0 >> + btr eax, 0 ; Set PE=0 >> + mov cr0, eax ; Write CR0 >> + >> + pop ax ; Discard alignment data >> + >> + ; >> + ; Clear registers (except RDX and RSP) before going into 16-bit mode >> + ; >> + xor eax, eax >> + xor ebx, ebx >> + xor ecx, ecx >> + xor esi, esi >> + xor edi, edi >> + xor ebp, ebp >> + >> + iret >> + >> +SwitchToRealProcEnd: >> + >> >> ;------------------------------------------------------------------------------------- >> ; AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, >> TopOfApStack, CountTofinish); >> >> ;------------------------------------------------------------------------------------- >> global ASM_PFX(AsmRelocateApLoop) >> ASM_PFX(AsmRelocateApLoop): >> AsmRelocateApLoopStart: >> +BITS 64 >> cli ; Disable interrupt before switching to >> 32-bit mode >> mov rax, [rsp + 40] ; CountTofinish >> lock dec dword [rax] ; (*CountTofinish)-- >> @@ -324,6 +558,11 @@ ASM_PFX(AsmGetAddressMap): >> mov qword [rcx + 18h], rax >> mov qword [rcx + 20h], AsmRelocateApLoopEnd - >> AsmRelocateApLoopStart >> mov qword [rcx + 28h], Flat32Start - RendezvousFunnelProcStart >> + mov qword [rcx + 30h], SwitchToRealProcEnd - >> SwitchToRealProcStart ; SwitchToRealSize >> + mov qword [rcx + 38h], SwitchToRealProcStart - >> RendezvousFunnelProcStart ; SwitchToRealOffset >> + mov qword [rcx + 40h], SwitchToRealProcStart - Flat32Start >> ; SwitchToRealNoNxOffset >> + mov qword [rcx + 48h], PM16Mode - RendezvousFunnelProcStart >> ; SwitchToRealPM16ModeOffset >> + mov qword [rcx + 50h], SwitchToRealProcEnd - PM16Mode >> ; SwitchToRealPM16ModeSize >> ret >> >> >> ;------------------------------------------------------------------------------------- >> > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#54048): https://edk2.groups.io/g/devel/message/54048 Mute This Topic: https://groups.io/mt/70985008/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-