https://github.com/freaknbigpanda created https://github.com/llvm/llvm-project/pull/210349
https://github.com/llvm/llvm-project/pull/193298 was reverted because of a compile time regression noticed in x86_64 (see results [here](https://llvm-compile-time-tracker.com/compare.php?from=b9869c8c920a7dfa983e215bc30729b13d8b353b&to=464b46a73b7592c95222cf2e64b34ffe6f2cf251&stat=instructions:u)) I added a conditional statement so that getFunctionFeatureMap and the subsequent string lookup is only called if FD->hasAttr<TargetAttr>() evaluates to true which should be very rare. >From 982d16b704a95c532050e8aba365d2ed71570c73 Mon Sep 17 00:00:00 2001 From: Benjamin Luke <[email protected]> Date: Thu, 16 Jul 2026 18:21:35 -0700 Subject: [PATCH 1/2] [clang][CodeGen][X86_64] Honor per-function AVX ABI in C/C++ call paths, maintain old psABI for PlayStation. (#193298) Fixes https://github.com/llvm/llvm-project/issues/64706 Wire per function x86 AVX ABI level into CodeGen arrangement methods so __attribute(__target("avx"))) / __attribute(__target("avx512f"))) on methods, ctors, and free-functions affects ABI lowering consistently. Specifically: - Added X86AVXABILevel member to CGFunctionInfo. - Populated X86AVXABILevel member in CGFunctionInfo objects via arrangement methods declared in CodeGenTypes.h. - Respect CGFunctionInfo AVX Level in X86_64ABIInfo::computeInfo. - Add/extend regression tests for: - free-function target-attribute AVX ABI lowering - C++ method/ctor target-attribute AVX ABI lowering - PS4/PS5 legacy ABI behavior (no per-function AVX ABI change) --------- Co-authored-by: Aaron Ballman <[email protected]> --- clang/docs/ReleaseNotes.md | 7 + clang/include/clang/Basic/ABIVersions.def | 6 + clang/include/clang/Basic/LangOptions.def | 2 +- clang/include/clang/CodeGen/CGFunctionInfo.h | 15 +- clang/include/clang/CodeGen/CodeGenABITypes.h | 12 +- clang/lib/CodeGen/ABIInfo.h | 9 + clang/lib/CodeGen/CGCUDARuntime.cpp | 5 +- clang/lib/CodeGen/CGCall.cpp | 156 ++++++++++++------ clang/lib/CodeGen/CGClass.cpp | 9 +- clang/lib/CodeGen/CGDeclCXX.cpp | 3 +- clang/lib/CodeGen/CGExpr.cpp | 2 +- clang/lib/CodeGen/CGExprCXX.cpp | 13 +- clang/lib/CodeGen/CGExprComplex.cpp | 3 +- clang/lib/CodeGen/CGHLSLRuntime.cpp | 6 +- clang/lib/CodeGen/CGObjCRuntime.cpp | 4 +- clang/lib/CodeGen/CGVTables.cpp | 6 +- clang/lib/CodeGen/CodeGenABITypes.cpp | 12 +- clang/lib/CodeGen/CodeGenFunction.cpp | 9 +- clang/lib/CodeGen/CodeGenFunction.h | 1 + clang/lib/CodeGen/CodeGenTypes.h | 37 +++-- clang/lib/CodeGen/ItaniumCXXABI.cpp | 2 +- clang/lib/CodeGen/MicrosoftCXXABI.cpp | 6 +- clang/lib/CodeGen/Targets/X86.cpp | 55 +++++- clang/test/CodeGen/sysv_abi.c | 18 ++ clang/test/CodeGen/target-avx-function-abi.c | 71 ++++++++ .../test/CodeGenCXX/target-avx-method-abi.cpp | 121 ++++++++++++++ .../unittests/CodeGen/CodeGenExternalTest.cpp | 145 ++++++++++++++-- clang/unittests/CodeGen/TestCompiler.h | 16 +- 28 files changed, 627 insertions(+), 124 deletions(-) create mode 100644 clang/test/CodeGen/target-avx-function-abi.c create mode 100644 clang/test/CodeGenCXX/target-avx-method-abi.cpp diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md index 8e47adbc64a4c..35cd71062a35f 100644 --- a/clang/docs/ReleaseNotes.md +++ b/clang/docs/ReleaseNotes.md @@ -50,6 +50,13 @@ latest release, please see the [Clang Web Site](https://clang.llvm.org) or the ### ABI Changes in This Version +- Except on PlayStation, Clang now derives the x86-64 System V AVX ABI level +for 256- and 512-bit vector arguments and returns from effective per-function +target features. Features and `arch=` CPUs that imply AVX or AVX512F are +honored, and calls use the caller's features, matching GCC. Per-function +features cannot lower the translation-unit ABI level; +`-fclang-abi-compat=23` restores the previous behavior. (#GH193298) + ### AST Dumping Potentially Breaking Changes ### Clang Frontend Potentially Breaking Changes diff --git a/clang/include/clang/Basic/ABIVersions.def b/clang/include/clang/Basic/ABIVersions.def index a9f72481ee0ea..15fb8ddf6d774 100644 --- a/clang/include/clang/Basic/ABIVersions.def +++ b/clang/include/clang/Basic/ABIVersions.def @@ -140,6 +140,12 @@ ABI_VER_MAJOR(21) /// local classes as nested-name entities instead of local-name entities. ABI_VER_MAJOR(22) +/// Attempt to be ABI-compatible with code generated by Clang 23.0.x. +/// This causes clang to: +/// - Ignore per-function target attributes when determining the x86 AVX ABI +/// level. +ABI_VER_MAJOR(23) + /// Conform to the underlying platform's C and C++ ABIs as closely as we can. ABI_VER_LATEST(Latest) diff --git a/clang/include/clang/Basic/LangOptions.def b/clang/include/clang/Basic/LangOptions.def index 1fb491a54a278..922124b696d43 100644 --- a/clang/include/clang/Basic/LangOptions.def +++ b/clang/include/clang/Basic/LangOptions.def @@ -426,7 +426,7 @@ LANGOPT(ForceEmitVTables, 1, 0, NotCompatible, "whether to emit all vtables") LANGOPT(AllowEditorPlaceholders, 1, 0, Benign, "allow editor placeholders in source") -ENUM_LANGOPT(ClangABICompat, ClangABI, 4, ClangABI::Latest, NotCompatible, +ENUM_LANGOPT(ClangABICompat, ClangABI, 5, ClangABI::Latest, NotCompatible, "version of Clang that we should attempt to be ABI-compatible " "with") diff --git a/clang/include/clang/CodeGen/CGFunctionInfo.h b/clang/include/clang/CodeGen/CGFunctionInfo.h index 713b52a4cc2b8..d1fc80337d859 100644 --- a/clang/include/clang/CodeGen/CGFunctionInfo.h +++ b/clang/include/clang/CodeGen/CGFunctionInfo.h @@ -17,10 +17,9 @@ #include "clang/AST/CanonicalType.h" #include "clang/AST/CharUnits.h" -#include "clang/AST/Decl.h" #include "clang/AST/Type.h" -#include "llvm/IR/DerivedTypes.h" #include "llvm/ADT/FoldingSet.h" +#include "llvm/IR/DerivedTypes.h" #include "llvm/Support/TrailingObjects.h" #include <cassert> @@ -653,6 +652,10 @@ class CGFunctionInfo final /// Log 2 of the maximum vector width. unsigned MaxVectorWidth : 4; + /// X86 AVX Level, can be different from global / module level AVX level + /// because of target attributes. + unsigned X86ABIAVXLevel = 0; + RequiredArgs Required; /// The struct representing all arguments passed in memory. Only used when @@ -683,7 +686,8 @@ class CGFunctionInfo final public: static CGFunctionInfo * create(unsigned llvmCC, bool instanceMethod, bool chainCall, - bool delegateCall, const FunctionType::ExtInfo &extInfo, + bool delegateCall, unsigned X86ABIAVXLevel, + const FunctionType::ExtInfo &extInfo, ArrayRef<ExtParameterInfo> paramInfos, CanQualType resultType, ArrayRef<CanQualType> argTypes, RequiredArgs required); void operator delete(void *p) { ::operator delete(p); } @@ -809,6 +813,8 @@ class CGFunctionInfo final MaxVectorWidth = llvm::countr_zero(Width) + 1; } + unsigned getX86ABIAVXLevel() const { return X86ABIAVXLevel; } + void Profile(llvm::FoldingSetNodeID &ID) { ID.AddInteger(getASTCallingConvention()); ID.AddBoolean(InstanceMethod); @@ -821,6 +827,7 @@ class CGFunctionInfo final ID.AddInteger(RegParm); ID.AddBoolean(NoCfCheck); ID.AddBoolean(CmseNSCall); + ID.AddInteger(X86ABIAVXLevel); ID.AddInteger(Required.getOpaqueData()); ID.AddBoolean(HasExtParameterInfos); if (HasExtParameterInfos) { @@ -833,6 +840,7 @@ class CGFunctionInfo final } static void Profile(llvm::FoldingSetNodeID &ID, bool InstanceMethod, bool ChainCall, bool IsDelegateCall, + unsigned X86ABIAVXLevel, const FunctionType::ExtInfo &info, ArrayRef<ExtParameterInfo> paramInfos, RequiredArgs required, CanQualType resultType, @@ -848,6 +856,7 @@ class CGFunctionInfo final ID.AddInteger(info.getRegParm()); ID.AddBoolean(info.getNoCfCheck()); ID.AddBoolean(info.getCmseNSCall()); + ID.AddInteger(X86ABIAVXLevel); ID.AddInteger(required.getOpaqueData()); ID.AddBoolean(!paramInfos.empty()); if (!paramInfos.empty()) { diff --git a/clang/include/clang/CodeGen/CodeGenABITypes.h b/clang/include/clang/CodeGen/CodeGenABITypes.h index 627dd0f0453ac..e79c31d3efa21 100644 --- a/clang/include/clang/CodeGen/CodeGenABITypes.h +++ b/clang/include/clang/CodeGen/CodeGenABITypes.h @@ -81,20 +81,22 @@ const CGFunctionInfo & arrangeCXXMethodCall(CodeGenModule &CGM, CanQualType returnType, ArrayRef<CanQualType> argTypes, FunctionType::ExtInfo info, ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, - RequiredArgs args); + RequiredArgs args, const FunctionDecl *CallerFD); const CGFunctionInfo &arrangeFreeFunctionCall( CodeGenModule &CGM, CanQualType returnType, ArrayRef<CanQualType> argTypes, FunctionType::ExtInfo info, - ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, - RequiredArgs args); + ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, RequiredArgs args, + const FunctionDecl *CallerFD); // An overload with an empty `paramInfos` inline const CGFunctionInfo & arrangeFreeFunctionCall(CodeGenModule &CGM, CanQualType returnType, ArrayRef<CanQualType> argTypes, - FunctionType::ExtInfo info, RequiredArgs args) { - return arrangeFreeFunctionCall(CGM, returnType, argTypes, info, {}, args); + FunctionType::ExtInfo info, RequiredArgs args, + const FunctionDecl *CallerFD) { + return arrangeFreeFunctionCall(CGM, returnType, argTypes, info, {}, args, + CallerFD); } /// Returns the implicit arguments to add to a complete, non-delegating C++ diff --git a/clang/lib/CodeGen/ABIInfo.h b/clang/lib/CodeGen/ABIInfo.h index 4563154d50960..b9a970919740f 100644 --- a/clang/lib/CodeGen/ABIInfo.h +++ b/clang/lib/CodeGen/ABIInfo.h @@ -26,6 +26,7 @@ class FixedVectorType; namespace clang { class ASTContext; class CodeGenOptions; +class FunctionDecl; class TargetInfo; namespace CodeGen { @@ -69,6 +70,14 @@ class ABIInfo { /// functions. llvm::CallingConv::ID getRuntimeCC() const { return RuntimeCC; } + // Get X86ABIAVXLevel for the given FunctionDecl and ExtInfo. + // This can be different than the global / module level X86ABIAVXLevel + // due to function attributes. + virtual unsigned getX86ABIAVXLevel(const FunctionDecl *, + const FunctionType::ExtInfo &) const { + return 0; + } + virtual void computeInfo(CodeGen::CGFunctionInfo &FI) const = 0; /// EmitVAArg - Emit the target dependent code to load a value of diff --git a/clang/lib/CodeGen/CGCUDARuntime.cpp b/clang/lib/CodeGen/CGCUDARuntime.cpp index 9c831b26c3a7b..0b09546453138 100644 --- a/clang/lib/CodeGen/CGCUDARuntime.cpp +++ b/clang/lib/CodeGen/CGCUDARuntime.cpp @@ -53,7 +53,8 @@ static llvm::Value *emitGetParamBuf(CodeGenFunction &CGF, Args.add(RValue::get(CGF.CGM.getSize(Offset)), CGF.getContext().getSizeType()); const CGFunctionInfo &CallInfo = CGF.CGM.getTypes().arrangeFreeFunctionCall( - Args, GetParamBufProto, /*ChainCall=*/false); + Args, GetParamBufProto, /*ChainCall=*/false, + CGF.getCurrentFunctionDecl()); auto Ret = CGF.EmitCall(CallInfo, Callee, /*ReturnValue=*/{}, Args); return Ret.getScalarVal(); @@ -116,7 +117,7 @@ RValue CGCUDARuntime::EmitCUDADeviceKernelCallExpr( LaunchCallArgs[1] = CallArg(RValue::get(Config), CGM.getContext().VoidPtrTy); const CGFunctionInfo &LaunchCallInfo = CGM.getTypes().arrangeFreeFunctionCall( LaunchCallArgs, LaunchCalleeFuncTy->getAs<FunctionProtoType>(), - /*ChainCall=*/false); + /*ChainCall=*/false, CGF.getCurrentFunctionDecl()); CGF.EmitCall(LaunchCallInfo, LaunchCallee, ReturnValue, LaunchCallArgs, CallOrInvoke, /*IsMustTail=*/false, E->getExprLoc()); diff --git a/clang/lib/CodeGen/CGCall.cpp b/clang/lib/CodeGen/CGCall.cpp index 08cb9860f2f92..a29f7aab0e58b 100644 --- a/clang/lib/CodeGen/CGCall.cpp +++ b/clang/lib/CodeGen/CGCall.cpp @@ -168,7 +168,7 @@ CodeGenTypes::arrangeFreeFunctionType(CanQual<FunctionNoProtoType> FTNP) { // variadic type. return arrangeLLVMFunctionInfo(FTNP->getReturnType().getUnqualifiedType(), FnInfoOpts::None, {}, FTNP->getExtInfo(), {}, - RequiredArgs(0)); + RequiredArgs(0), /*ABIInfoFD=*/nullptr); } static void addExtParameterInfosForCall( @@ -246,7 +246,8 @@ arrangeLLVMFunctionInfo(CodeGenTypes &CGT, bool instanceMethod, FnInfoOpts opts = instanceMethod ? FnInfoOpts::IsInstanceMethod : FnInfoOpts::None; return CGT.arrangeLLVMFunctionInfo(resultType, opts, prefix, - FTP->getExtInfo(), paramInfos, Required); + FTP->getExtInfo(), paramInfos, Required, + /*ABIInfoFD=*/nullptr); } using CanQualTypeList = SmallVector<CanQualType, 16>; @@ -357,10 +358,16 @@ CodeGenTypes::arrangeCXXMethodType(const CXXRecordDecl *RD, // Add the 'this' pointer. argTypes.push_back(DeriveThisType(RD, MD)); - - return ::arrangeLLVMFunctionInfo( - *this, /*instanceMethod=*/true, argTypes, - FTP->getCanonicalTypeUnqualified().getAs<FunctionProtoType>()); + auto CanonicalFTP = + FTP->getCanonicalTypeUnqualified().getAs<FunctionProtoType>(); + ExtParameterInfoList paramInfos; + RequiredArgs required = RequiredArgs::forPrototypePlus( + CanonicalFTP.getTypePtr(), argTypes.size()); + appendParameterTypes(*this, argTypes, paramInfos, CanonicalFTP); + return arrangeLLVMFunctionInfo( + CanonicalFTP->getReturnType().getUnqualifiedType(), + FnInfoOpts::IsInstanceMethod, argTypes, CanonicalFTP->getExtInfo(), + paramInfos, required, MD); } /// Set calling convention for CUDA/HIP kernel. @@ -393,7 +400,13 @@ CodeGenTypes::arrangeCXXMethodDeclaration(const CXXMethodDecl *MD) { return arrangeCXXMethodType(ThisType, prototype.getTypePtr(), MD); } - return arrangeFreeFunctionType(prototype); + CanQualTypeList argTypes; + ExtParameterInfoList paramInfos; + appendParameterTypes(*this, argTypes, paramInfos, prototype); + return arrangeLLVMFunctionInfo( + prototype->getReturnType().getUnqualifiedType(), FnInfoOpts::None, + argTypes, prototype->getExtInfo(), paramInfos, + RequiredArgs::forPrototypePlus(prototype.getTypePtr(), 0), MD); } bool CodeGenTypes::inheritingCtorHasParams( @@ -452,7 +465,7 @@ CodeGenTypes::arrangeCXXStructorDeclaration(GlobalDecl GD) { ? CGM.getContext().VoidPtrTy : Context.VoidTy; return arrangeLLVMFunctionInfo(resultType, FnInfoOpts::IsInstanceMethod, - argTypes, extInfo, paramInfos, required); + argTypes, extInfo, paramInfos, required, MD); } static CanQualTypeList getArgTypesForCall(ASTContext &ctx, @@ -491,7 +504,8 @@ getExtParameterInfosForCall(const FunctionProtoType *proto, unsigned prefixArgs, /// given CXXConstructorDecl. const CGFunctionInfo &CodeGenTypes::arrangeCXXConstructorCall( const CallArgList &args, const CXXConstructorDecl *D, CXXCtorType CtorKind, - unsigned ExtraPrefixArgs, unsigned ExtraSuffixArgs, bool PassProtoArgs) { + unsigned ExtraPrefixArgs, unsigned ExtraSuffixArgs, + const FunctionDecl *ABIInfoFD, bool PassProtoArgs) { CanQualTypeList ArgTypes; for (const auto &Arg : args) ArgTypes.push_back(Context.getCanonicalParamType(Arg.Ty)); @@ -522,7 +536,8 @@ const CGFunctionInfo &CodeGenTypes::arrangeCXXConstructorCall( } return arrangeLLVMFunctionInfo(ResultType, FnInfoOpts::IsInstanceMethod, - ArgTypes, Info, ParamInfos, Required); + ArgTypes, Info, ParamInfos, Required, + ABIInfoFD); } /// Arrange the argument and result information for the declaration or @@ -551,10 +566,17 @@ CodeGenTypes::arrangeFunctionDeclaration(const GlobalDecl GD) { if (CanQual<FunctionNoProtoType> noProto = FTy.getAs<FunctionNoProtoType>()) { return arrangeLLVMFunctionInfo(noProto->getReturnType(), FnInfoOpts::None, {}, noProto->getExtInfo(), {}, - RequiredArgs::All); + RequiredArgs::All, FD); } - return arrangeFreeFunctionType(FTy.castAs<FunctionProtoType>()); + CanQual<FunctionProtoType> FTP = FTy.castAs<FunctionProtoType>(); + CanQualTypeList argTypes; + ExtParameterInfoList paramInfos; + appendParameterTypes(*this, argTypes, paramInfos, FTP); + return arrangeLLVMFunctionInfo(FTP->getReturnType().getUnqualifiedType(), + FnInfoOpts::None, argTypes, FTP->getExtInfo(), + paramInfos, + RequiredArgs::forPrototypePlus(FTP, 0), FD); } /// Arrange the argument and result information for the declaration or @@ -603,7 +625,7 @@ CodeGenTypes::arrangeObjCMessageSendSignature(const ObjCMethodDecl *MD, return arrangeLLVMFunctionInfo(GetReturnType(MD->getReturnType()), FnInfoOpts::None, argTys, einfo, extParamInfos, - required); + required, /*ABIInfoFD=*/nullptr); } const CGFunctionInfo & @@ -613,7 +635,8 @@ CodeGenTypes::arrangeUnprototypedObjCMessageSend(QualType returnType, FunctionType::ExtInfo einfo; return arrangeLLVMFunctionInfo(GetReturnType(returnType), FnInfoOpts::None, - argTypes, einfo, {}, RequiredArgs::All); + argTypes, einfo, {}, RequiredArgs::All, + nullptr); } const CGFunctionInfo &CodeGenTypes::arrangeGlobalDeclaration(GlobalDecl GD) { @@ -636,7 +659,7 @@ CodeGenTypes::arrangeUnprototypedMustTailThunk(const CXXMethodDecl *MD) { CanQual<FunctionProtoType> FTP = GetFormalType(MD); CanQualType ArgTys[] = {DeriveThisType(MD->getParent(), MD)}; return arrangeLLVMFunctionInfo(Context.VoidTy, FnInfoOpts::None, ArgTys, - FTP->getExtInfo(), {}, RequiredArgs(1)); + FTP->getExtInfo(), {}, RequiredArgs(1), MD); } const CGFunctionInfo & @@ -656,7 +679,7 @@ CodeGenTypes::arrangeMSCtorClosure(const CXXConstructorDecl *CD, /*IsVariadic=*/false, /*IsCXXMethod=*/true); return arrangeLLVMFunctionInfo(Context.VoidTy, FnInfoOpts::IsInstanceMethod, ArgTys, FunctionType::ExtInfo(CC), {}, - RequiredArgs::All); + RequiredArgs::All, /*ABIInfoFD=*/nullptr); } /// Arrange a call as unto a free function, except possibly with an @@ -664,7 +687,8 @@ CodeGenTypes::arrangeMSCtorClosure(const CXXConstructorDecl *CD, static const CGFunctionInfo & arrangeFreeFunctionLikeCall(CodeGenTypes &CGT, CodeGenModule &CGM, const CallArgList &args, const FunctionType *fnType, - unsigned numExtraRequiredArgs, bool chainCall) { + unsigned numExtraRequiredArgs, bool chainCall, + const FunctionDecl *ABIInfoFD) { assert(args.size() >= numExtraRequiredArgs); ExtParameterInfoList paramInfos; @@ -697,7 +721,7 @@ arrangeFreeFunctionLikeCall(CodeGenTypes &CGT, CodeGenModule &CGM, FnInfoOpts opts = chainCall ? FnInfoOpts::IsChainCall : FnInfoOpts::None; return CGT.arrangeLLVMFunctionInfo(GetReturnType(fnType->getReturnType()), opts, argTypes, fnType->getExtInfo(), - paramInfos, required); + paramInfos, required, ABIInfoFD); } /// Figure out the rules for calling a function with the given formal @@ -705,9 +729,10 @@ arrangeFreeFunctionLikeCall(CodeGenTypes &CGT, CodeGenModule &CGM, /// because the function might be unprototyped, in which case it's /// target-dependent in crazy ways. const CGFunctionInfo &CodeGenTypes::arrangeFreeFunctionCall( - const CallArgList &args, const FunctionType *fnType, bool chainCall) { + const CallArgList &args, const FunctionType *fnType, bool chainCall, + const FunctionDecl *ABIInfoFD) { return arrangeFreeFunctionLikeCall(*this, CGM, args, fnType, - chainCall ? 1 : 0, chainCall); + chainCall ? 1 : 0, chainCall, ABIInfoFD); } /// A block function is essentially a free function with an @@ -715,8 +740,10 @@ const CGFunctionInfo &CodeGenTypes::arrangeFreeFunctionCall( const CGFunctionInfo & CodeGenTypes::arrangeBlockFunctionCall(const CallArgList &args, const FunctionType *fnType) { + // FIXME: Pass the enclosing function's ABI information so block calls use + // the caller's target features. return arrangeFreeFunctionLikeCall(*this, CGM, args, fnType, 1, - /*chainCall=*/false); + /*chainCall=*/false, nullptr); } const CGFunctionInfo & @@ -726,10 +753,11 @@ CodeGenTypes::arrangeBlockFunctionDeclaration(const FunctionProtoType *proto, getExtParameterInfosForCall(proto, 1, params.size()); CanQualTypeList argTypes = getArgTypesForDeclaration(Context, params); - return arrangeLLVMFunctionInfo(GetReturnType(proto->getReturnType()), - FnInfoOpts::None, argTypes, - proto->getExtInfo(), paramInfos, - RequiredArgs::forPrototypePlus(proto, 1)); + // FIXME: Use the block's target features when arranging its invoke function. + return arrangeLLVMFunctionInfo( + GetReturnType(proto->getReturnType()), FnInfoOpts::None, argTypes, + proto->getExtInfo(), paramInfos, RequiredArgs::forPrototypePlus(proto, 1), + /*ABIInfoFD=*/nullptr); } const CGFunctionInfo & @@ -740,7 +768,7 @@ CodeGenTypes::arrangeBuiltinFunctionCall(QualType resultType, argTypes.push_back(Context.getCanonicalParamType(Arg.Ty)); return arrangeLLVMFunctionInfo(GetReturnType(resultType), FnInfoOpts::None, argTypes, FunctionType::ExtInfo(), - /*paramInfos=*/{}, RequiredArgs::All); + /*paramInfos=*/{}, RequiredArgs::All, nullptr); } const CGFunctionInfo & @@ -750,14 +778,14 @@ CodeGenTypes::arrangeBuiltinFunctionDeclaration(QualType resultType, return arrangeLLVMFunctionInfo(GetReturnType(resultType), FnInfoOpts::None, argTypes, FunctionType::ExtInfo(), {}, - RequiredArgs::All); + RequiredArgs::All, /*ABIInfoFD=*/nullptr); } const CGFunctionInfo &CodeGenTypes::arrangeBuiltinFunctionDeclaration( CanQualType resultType, ArrayRef<CanQualType> argTypes) { return arrangeLLVMFunctionInfo(resultType, FnInfoOpts::None, argTypes, - FunctionType::ExtInfo(), {}, - RequiredArgs::All); + FunctionType::ExtInfo(), {}, RequiredArgs::All, + /*ABIInfoFD=*/nullptr); } const CGFunctionInfo &CodeGenTypes::arrangeDeviceKernelCallerDeclaration( @@ -767,7 +795,8 @@ const CGFunctionInfo &CodeGenTypes::arrangeDeviceKernelCallerDeclaration( return arrangeLLVMFunctionInfo(GetReturnType(resultType), FnInfoOpts::None, argTypes, FunctionType::ExtInfo(CC_DeviceKernel), - /*paramInfos=*/{}, RequiredArgs::All); + /*paramInfos=*/{}, RequiredArgs::All, + /*ABIInfoFD=*/nullptr); } /// Arrange a call to a C++ method, passing the given arguments. @@ -776,7 +805,8 @@ const CGFunctionInfo &CodeGenTypes::arrangeDeviceKernelCallerDeclaration( /// does not count `this`. const CGFunctionInfo &CodeGenTypes::arrangeCXXMethodCall( const CallArgList &args, const FunctionProtoType *proto, - RequiredArgs required, unsigned numPrefixArgs) { + RequiredArgs required, unsigned numPrefixArgs, + const FunctionDecl *ABIInfoFD) { assert(numPrefixArgs + 1 <= args.size() && "Emitting a call with less args than the required prefix?"); // Add one to account for `this`. It's a bit awkward here, but we don't count @@ -789,19 +819,23 @@ const CGFunctionInfo &CodeGenTypes::arrangeCXXMethodCall( FunctionType::ExtInfo info = proto->getExtInfo(); return arrangeLLVMFunctionInfo(GetReturnType(proto->getReturnType()), FnInfoOpts::IsInstanceMethod, argTypes, info, - paramInfos, required); + paramInfos, required, ABIInfoFD); } const CGFunctionInfo &CodeGenTypes::arrangeNullaryFunction() { return arrangeLLVMFunctionInfo(getContext().VoidTy, FnInfoOpts::None, {}, - FunctionType::ExtInfo(), {}, - RequiredArgs::All); + FunctionType::ExtInfo(), {}, RequiredArgs::All, + /*ABIInfoFD=*/nullptr); } const CGFunctionInfo &CodeGenTypes::arrangeCall(const CGFunctionInfo &signature, - const CallArgList &args) { + const CallArgList &args, + const FunctionDecl *ABIInfoFD) { assert(signature.arg_size() <= args.size()); - if (signature.arg_size() == args.size()) + unsigned X86ABIAVXLevel = + CGM.getABIInfo().getX86ABIAVXLevel(ABIInfoFD, signature.getExtInfo()); + if (signature.arg_size() == args.size() && + signature.getX86ABIAVXLevel() == X86ABIAVXLevel) return signature; ExtParameterInfoList paramInfos; @@ -821,9 +855,13 @@ const CGFunctionInfo &CodeGenTypes::arrangeCall(const CGFunctionInfo &signature, opts |= FnInfoOpts::IsChainCall; if (signature.isDelegateCall()) opts |= FnInfoOpts::IsDelegateCall; - return arrangeLLVMFunctionInfo(signature.getReturnType(), opts, argTypes, - signature.getExtInfo(), paramInfos, - signature.getRequiredArgs()); + + const CGFunctionInfo *newFI = findOrInsertCGFunctionInfo( + signature.isInstanceMethod(), signature.isChainCall(), + signature.isDelegateCall(), X86ABIAVXLevel, signature.getExtInfo(), + paramInfos, signature.getRequiredArgs(), signature.getReturnType(), + argTypes); + return *newFI; } namespace clang { @@ -1019,7 +1057,7 @@ const CGFunctionInfo &CodeGenTypes::arrangeLLVMFunctionInfo( CanQualType resultType, FnInfoOpts opts, ArrayRef<CanQualType> argTypes, FunctionType::ExtInfo info, ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, - RequiredArgs required) { + RequiredArgs required, const FunctionDecl *ABIInfoFD) { assert(llvm::all_of(argTypes, [](CanQualType T) { return T.isCanonicalAsParam(); })); @@ -1031,19 +1069,36 @@ const CGFunctionInfo &CodeGenTypes::arrangeLLVMFunctionInfo( (opts & FnInfoOpts::IsChainCall) == FnInfoOpts::IsChainCall; bool isDelegateCall = (opts & FnInfoOpts::IsDelegateCall) == FnInfoOpts::IsDelegateCall; + unsigned X86ABIAVXLevel = CGM.getABIInfo().getX86ABIAVXLevel(ABIInfoFD, info); + + const CGFunctionInfo *newFI = findOrInsertCGFunctionInfo( + isInstanceMethod, isChainCall, isDelegateCall, X86ABIAVXLevel, info, + paramInfos, required, resultType, argTypes); + return *newFI; +} + +CGFunctionInfo *CodeGenTypes::findOrInsertCGFunctionInfo( + bool isInstanceMethod, bool isChainCall, bool isDelegateCall, + unsigned X86ABIAVXLevel, const FunctionType::ExtInfo &info, + ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, + RequiredArgs required, CanQualType resultType, + ArrayRef<CanQualType> argTypes) { + llvm::FoldingSetNodeID ID; CGFunctionInfo::Profile(ID, isInstanceMethod, isChainCall, isDelegateCall, - info, paramInfos, required, resultType, argTypes); + X86ABIAVXLevel, info, paramInfos, required, + resultType, argTypes); void *insertPos = nullptr; CGFunctionInfo *FI = FunctionInfos.FindNodeOrInsertPos(ID, insertPos); if (FI) - return *FI; + return FI; unsigned CC = ClangCallConvToLLVMCallConv(info.getCC()); // Construct the function info. We co-allocate the ArgInfos. FI = CGFunctionInfo::create(CC, isInstanceMethod, isChainCall, isDelegateCall, - info, paramInfos, resultType, argTypes, required); + X86ABIAVXLevel, info, paramInfos, resultType, + argTypes, required); FunctionInfos.InsertNode(FI, insertPos); bool inserted = FunctionsBeingProcessed.insert(FI).second; @@ -1082,16 +1137,14 @@ const CGFunctionInfo &CodeGenTypes::arrangeLLVMFunctionInfo( (void)erased; assert(erased && "Not in set?"); - return *FI; + return FI; } -CGFunctionInfo *CGFunctionInfo::create(unsigned llvmCC, bool instanceMethod, - bool chainCall, bool delegateCall, - const FunctionType::ExtInfo &info, - ArrayRef<ExtParameterInfo> paramInfos, - CanQualType resultType, - ArrayRef<CanQualType> argTypes, - RequiredArgs required) { +CGFunctionInfo *CGFunctionInfo::create( + unsigned llvmCC, bool instanceMethod, bool chainCall, bool delegateCall, + unsigned X86ABIAVXLevel, const FunctionType::ExtInfo &info, + ArrayRef<ExtParameterInfo> paramInfos, CanQualType resultType, + ArrayRef<CanQualType> argTypes, RequiredArgs required) { assert(paramInfos.empty() || paramInfos.size() == argTypes.size()); assert(!required.allowsOptionalArgs() || required.getNumRequiredArgs() <= argTypes.size()); @@ -1114,6 +1167,7 @@ CGFunctionInfo *CGFunctionInfo::create(unsigned llvmCC, bool instanceMethod, FI->Required = required; FI->HasRegParm = info.getHasRegParm(); FI->RegParm = info.getRegParm(); + FI->X86ABIAVXLevel = X86ABIAVXLevel; FI->ArgStruct = nullptr; FI->ArgStructAlign = 0; FI->NumArgs = argTypes.size(); diff --git a/clang/lib/CodeGen/CGClass.cpp b/clang/lib/CodeGen/CGClass.cpp index 39b8e50f68eaf..9c0c1cbdeb219 100644 --- a/clang/lib/CodeGen/CGClass.cpp +++ b/clang/lib/CodeGen/CGClass.cpp @@ -2339,8 +2339,8 @@ static bool canEmitDelegateCallArgs(CodeGenFunction &CGF, return false; // Likewise if they're inalloca. - const CGFunctionInfo &Info = - CGF.CGM.getTypes().arrangeCXXConstructorCall(Args, Ctor, Type, 0, 0); + const CGFunctionInfo &Info = CGF.CGM.getTypes().arrangeCXXConstructorCall( + Args, Ctor, Type, 0, 0, CGF.getCurrentFunctionDecl()); if (Info.usesInAlloca()) return false; } @@ -2400,7 +2400,8 @@ void CodeGenFunction::EmitCXXConstructorCall( // Emit the call. llvm::Constant *CalleePtr = CGM.getAddrOfCXXStructor(GlobalDecl(D, Type)); const CGFunctionInfo &Info = CGM.getTypes().arrangeCXXConstructorCall( - Args, D, Type, ExtraArgs.Prefix, ExtraArgs.Suffix, PassPrototypeArgs); + Args, D, Type, ExtraArgs.Prefix, ExtraArgs.Suffix, + getCurrentFunctionDecl(), PassPrototypeArgs); CGCallee Callee = CGCallee::forDirect(CalleePtr, GlobalDecl(D, Type)); EmitCall(Info, Callee, ReturnValueSlot(), Args, CallOrInvoke, false, Loc); @@ -3288,7 +3289,7 @@ void CodeGenFunction::EmitLambdaInAllocaImplFn( ArgTypes.push_back(I->type); *ImplFnInfo = &CGM.getTypes().arrangeLLVMFunctionInfo( FnInfo.getReturnType(), FnInfoOpts::IsDelegateCall, ArgTypes, - FnInfo.getExtInfo(), {}, FnInfo.getRequiredArgs()); + FnInfo.getExtInfo(), {}, FnInfo.getRequiredArgs(), CallOp); // Create mangled name as if this was a method named __impl. If for some // reason the name doesn't look as expected then just tack __impl to the diff --git a/clang/lib/CodeGen/CGDeclCXX.cpp b/clang/lib/CodeGen/CGDeclCXX.cpp index 57c6bad67dab9..a54809c48037e 100644 --- a/clang/lib/CodeGen/CGDeclCXX.cpp +++ b/clang/lib/CodeGen/CGDeclCXX.cpp @@ -286,7 +286,8 @@ llvm::Function *CodeGenFunction::createTLSAtExitStub( const CGFunctionInfo &FI = CGM.getTypes().arrangeLLVMFunctionInfo( getContext().IntTy, FnInfoOpts::None, {getContext().IntTy}, - FunctionType::ExtInfo(), {}, RequiredArgs::All); + FunctionType::ExtInfo(), {}, RequiredArgs::All, + /*ABIInfoFD=*/nullptr); // Get the stub function type, int(*)(int,...). llvm::FunctionType *StubTy = diff --git a/clang/lib/CodeGen/CGExpr.cpp b/clang/lib/CodeGen/CGExpr.cpp index dd42e9550316d..9201e40bc13a1 100644 --- a/clang/lib/CodeGen/CGExpr.cpp +++ b/clang/lib/CodeGen/CGExpr.cpp @@ -7141,7 +7141,7 @@ RValue CodeGenFunction::EmitCall(QualType CalleeType, E->getDirectCallee(), /*ParamsToSkip=*/0, Order); const CGFunctionInfo &FnInfo = CGM.getTypes().arrangeFreeFunctionCall( - Args, FnType, /*ChainCall=*/Chain); + Args, FnType, /*ChainCall=*/Chain, getCurrentFunctionDecl()); if (ResolvedFnInfo) *ResolvedFnInfo = &FnInfo; diff --git a/clang/lib/CodeGen/CGExprCXX.cpp b/clang/lib/CodeGen/CGExprCXX.cpp index 1769ab00ed56d..e400a5c5a49c5 100644 --- a/clang/lib/CodeGen/CGExprCXX.cpp +++ b/clang/lib/CodeGen/CGExprCXX.cpp @@ -92,7 +92,8 @@ RValue CodeGenFunction::EmitCXXMemberOrOperatorCall( MemberCallInfo CallInfo = commonEmitCXXMemberOrOperatorCall( *this, MD, This, ImplicitParam, ImplicitParamTy, CE, Args, RtlArgs); auto &FnInfo = CGM.getTypes().arrangeCXXMethodCall( - Args, FPT, CallInfo.ReqArgs, CallInfo.PrefixSize); + Args, FPT, CallInfo.ReqArgs, CallInfo.PrefixSize, + getCurrentFunctionDecl()); return EmitCall(FnInfo, Callee, ReturnValue, Args, CallOrInvoke, CE && CE == MustTailCall, CE ? CE->getExprLoc() : SourceLocation()); @@ -494,7 +495,8 @@ CodeGenFunction::EmitCXXMemberPointerCallExpr(const CXXMemberCallExpr *E, // And the rest of the call args EmitCallArgs(Args, FPT, E->arguments()); return EmitCall(CGM.getTypes().arrangeCXXMethodCall(Args, FPT, required, - /*PrefixSize=*/0), + /*PrefixSize=*/0, + getCurrentFunctionDecl()), Callee, ReturnValue, Args, CallOrInvoke, E == MustTailCall, E->getExprLoc()); } @@ -1354,9 +1356,10 @@ static RValue EmitNewDeleteCall(CodeGenFunction &CGF, llvm::Constant *CalleePtr = CalleeOverride ? CalleeOverride : CGF.CGM.GetAddrOfFunction(CalleeDecl); CGCallee Callee = CGCallee::forDirect(CalleePtr, GlobalDecl(CalleeDecl)); - RValue RV = CGF.EmitCall(CGF.CGM.getTypes().arrangeFreeFunctionCall( - Args, CalleeType, /*ChainCall=*/false), - Callee, ReturnValueSlot(), Args, &CallOrInvoke); + RValue RV = CGF.EmitCall( + CGF.CGM.getTypes().arrangeFreeFunctionCall( + Args, CalleeType, /*ChainCall=*/false, CGF.getCurrentFunctionDecl()), + Callee, ReturnValueSlot(), Args, &CallOrInvoke); /// C++1y [expr.new]p10: /// [In a new-expression,] an implementation is allowed to omit a call diff --git a/clang/lib/CodeGen/CGExprComplex.cpp b/clang/lib/CodeGen/CGExprComplex.cpp index 4d98ee9957418..350cbd18c7ed7 100644 --- a/clang/lib/CodeGen/CGExprComplex.cpp +++ b/clang/lib/CodeGen/CGExprComplex.cpp @@ -782,7 +782,8 @@ ComplexPairTy ComplexExprEmitter::EmitComplexBinOpLibCall(StringRef LibCallName, 4, Op.Ty->castAs<ComplexType>()->getElementType()); QualType FQTy = CGF.getContext().getFunctionType(Op.Ty, ArgsQTys, EPI); const CGFunctionInfo &FuncInfo = CGF.CGM.getTypes().arrangeFreeFunctionCall( - Args, cast<FunctionType>(FQTy.getTypePtr()), false); + Args, cast<FunctionType>(FQTy.getTypePtr()), false, + CGF.getCurrentFunctionDecl()); llvm::FunctionType *FTy = CGF.CGM.getTypes().GetFunctionType(FuncInfo); llvm::FunctionCallee Func = CGF.CGM.CreateRuntimeFunction( diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp index 8794579166b6a..ea5c1a24d726e 100644 --- a/clang/lib/CodeGen/CGHLSLRuntime.cpp +++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp @@ -402,8 +402,10 @@ static void callResourceInitMethod(CodeGenFunction &CGF, llvm::Constant *CalleeFn = CGF.CGM.GetAddrOfFunction(CreateMethod); const FunctionProtoType *Proto = CreateMethod->getType()->getAs<FunctionProtoType>(); - const CGFunctionInfo &FnInfo = - CGF.CGM.getTypes().arrangeFreeFunctionCall(Args, Proto, false); + // HLSL code generation is restricted to DXIL and SPIR-V targets, so no + // caller declaration is needed for x86 SysV ABI selection. + const CGFunctionInfo &FnInfo = CGF.CGM.getTypes().arrangeFreeFunctionCall( + Args, Proto, false, /*ABIInfoFD=*/nullptr); ReturnValueSlot ReturnValue(ReturnAddress, false); CGCallee Callee(CGCalleeInfo(Proto), CalleeFn); CGF.EmitCall(FnInfo, Callee, ReturnValue, Args, nullptr); diff --git a/clang/lib/CodeGen/CGObjCRuntime.cpp b/clang/lib/CodeGen/CGObjCRuntime.cpp index 2b8bd170b7197..a83a4ce67a9c6 100644 --- a/clang/lib/CodeGen/CGObjCRuntime.cpp +++ b/clang/lib/CodeGen/CGObjCRuntime.cpp @@ -364,13 +364,15 @@ CGObjCRuntime::getMessageSendInfo(const ObjCMethodDecl *method, llvm::PointerType *signatureType = llvm::PointerType::get(CGM.getLLVMContext(), ProgramAS); + // FIXME: Pass the enclosing FunctionDecl so Objective-C message sends use + // the caller's target features when arranging their ABI. // If there's a method, use information from that. if (method) { const CGFunctionInfo &signature = CGM.getTypes().arrangeObjCMessageSendSignature(method, callArgs[0].Ty); const CGFunctionInfo &signatureForCall = - CGM.getTypes().arrangeCall(signature, callArgs); + CGM.getTypes().arrangeCall(signature, callArgs, /*ABIInfoFD=*/nullptr); return MessageSendInfo(signatureForCall, signatureType); } diff --git a/clang/lib/CodeGen/CGVTables.cpp b/clang/lib/CodeGen/CGVTables.cpp index 9e4218646669c..f8cea57739949 100644 --- a/clang/lib/CodeGen/CGVTables.cpp +++ b/clang/lib/CodeGen/CGVTables.cpp @@ -383,10 +383,12 @@ void CodeGenFunction::EmitCallAndReturnForThunk(llvm::FunctionCallee Callee, #ifndef NDEBUG const CGFunctionInfo &CallFnInfo = CGM.getTypes().arrangeCXXMethodCall( - CallArgs, FPT, RequiredArgs::forPrototypePlus(FPT, 1), PrefixArgs); + CallArgs, FPT, RequiredArgs::forPrototypePlus(FPT, 1), PrefixArgs, MD); assert(CallFnInfo.getRegParm() == CurFnInfo->getRegParm() && CallFnInfo.isNoReturn() == CurFnInfo->isNoReturn() && - CallFnInfo.getCallingConvention() == CurFnInfo->getCallingConvention()); + CallFnInfo.getCallingConvention() == + CurFnInfo->getCallingConvention() && + CallFnInfo.getX86ABIAVXLevel() == CurFnInfo->getX86ABIAVXLevel()); assert(isa<CXXDestructorDecl>(MD) || // ignore dtor return types similar(CallFnInfo.getReturnInfo(), CallFnInfo.getReturnType(), CurFnInfo->getReturnInfo(), CurFnInfo->getReturnType())); diff --git a/clang/lib/CodeGen/CodeGenABITypes.cpp b/clang/lib/CodeGen/CodeGenABITypes.cpp index aad286f3de53c..a2f4c6fd9a26a 100644 --- a/clang/lib/CodeGen/CodeGenABITypes.cpp +++ b/clang/lib/CodeGen/CodeGenABITypes.cpp @@ -60,20 +60,20 @@ CodeGen::arrangeCXXMethodType(CodeGenModule &CGM, const CGFunctionInfo &CodeGen::arrangeCXXMethodCall( CodeGenModule &CGM, CanQualType returnType, ArrayRef<CanQualType> argTypes, FunctionType::ExtInfo info, - ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, - RequiredArgs args) { + ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, RequiredArgs args, + const FunctionDecl *CallerFD) { return CGM.getTypes().arrangeLLVMFunctionInfo( returnType, FnInfoOpts::IsInstanceMethod, argTypes, info, paramInfos, - args); + args, CallerFD); } const CGFunctionInfo &CodeGen::arrangeFreeFunctionCall( CodeGenModule &CGM, CanQualType returnType, ArrayRef<CanQualType> argTypes, FunctionType::ExtInfo info, - ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, - RequiredArgs args) { + ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, RequiredArgs args, + const FunctionDecl *CallerFD) { return CGM.getTypes().arrangeLLVMFunctionInfo( - returnType, FnInfoOpts::None, argTypes, info, paramInfos, args); + returnType, FnInfoOpts::None, argTypes, info, paramInfos, args, CallerFD); } ImplicitCXXConstructorArgs diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp index fc4c7ea40f03d..e1645fc06ed5a 100644 --- a/clang/lib/CodeGen/CodeGenFunction.cpp +++ b/clang/lib/CodeGen/CodeGenFunction.cpp @@ -91,6 +91,13 @@ CodeGenFunction::CodeGenFunction(CodeGenModule &cgm, bool suppressNewContext) SetFastMathFlags(CurFPFeatures); } +const FunctionDecl *CodeGenFunction::getCurrentFunctionDecl() const { + const auto *FD = dyn_cast_or_null<FunctionDecl>(CurCodeDecl); + if (!FD) + FD = dyn_cast_or_null<FunctionDecl>(CurFuncDecl); + return FD; +} + CodeGenFunction::~CodeGenFunction() { assert(LifetimeExtendedCleanupStack.empty() && "failed to emit a cleanup"); assert(DeferredDeactivationCleanupStack.empty() && @@ -1615,7 +1622,7 @@ void CodeGenFunction::GenerateCode(GlobalDecl GD, llvm::Function *Fn, const FunctionType *FT = cast<FunctionType>(FD->getType()); CGM.getTargetCodeGenInfo().setOCLKernelStubCallingConvention(FT); const CGFunctionInfo &FnInfo = CGM.getTypes().arrangeFreeFunctionCall( - CallArgs, FT, /*ChainCall=*/false); + CallArgs, FT, /*ChainCall=*/false, getCurrentFunctionDecl()); llvm::FunctionType *FTy = CGM.getTypes().GetFunctionType(FnInfo); llvm::Constant *GDStubFunctionPointer = CGM.getRawFunctionPointer(GDStub, FTy); diff --git a/clang/lib/CodeGen/CodeGenFunction.h b/clang/lib/CodeGen/CodeGenFunction.h index e7c24f1f36f1e..3fc9052adb87b 100644 --- a/clang/lib/CodeGen/CodeGenFunction.h +++ b/clang/lib/CodeGen/CodeGenFunction.h @@ -2236,6 +2236,7 @@ class CodeGenFunction : public CodeGenTypeCache { const TargetCodeGenInfo &getTargetHooks() const { return CGM.getTargetCodeGenInfo(); } + const FunctionDecl *getCurrentFunctionDecl() const; //===--------------------------------------------------------------------===// // Cleanups diff --git a/clang/lib/CodeGen/CodeGenTypes.h b/clang/lib/CodeGen/CodeGenTypes.h index 9de7e0a83579d..860f1f1ccdce2 100644 --- a/clang/lib/CodeGen/CodeGenTypes.h +++ b/clang/lib/CodeGen/CodeGenTypes.h @@ -33,6 +33,7 @@ template <typename> class CanQual; class CXXConstructorDecl; class CXXMethodDecl; class CodeGenOptions; +class FunctionDecl; class FunctionProtoType; class QualType; class RecordDecl; @@ -89,9 +90,18 @@ class CodeGenTypes { llvm::DenseMap<const Type *, llvm::Type *> RecordsWithOpaqueMemberPointers; static constexpr unsigned FunctionInfosLog2InitSize = 9; + /// Helper for ConvertType. llvm::Type *ConvertFunctionTypeInternal(QualType FT); + // Helper to insert CGFunctionInfo objects + CGFunctionInfo *findOrInsertCGFunctionInfo( + bool isInstanceMethod, bool isChainCall, bool isDelegateCall, + unsigned X86ABIAVXLevel, const FunctionType::ExtInfo &info, + ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, + RequiredArgs required, CanQualType resultType, + ArrayRef<CanQualType> argTypes); + public: CodeGenTypes(CodeGenModule &cgm); ~CodeGenTypes(); @@ -203,14 +213,16 @@ class CodeGenTypes { /// /// Often this will be able to simply return the declaration info. const CGFunctionInfo &arrangeCall(const CGFunctionInfo &declFI, - const CallArgList &args); + const CallArgList &args, + const FunctionDecl *ABIInfoFD); /// Free functions are functions that are compatible with an ordinary /// C function pointer type. const CGFunctionInfo &arrangeFunctionDeclaration(const GlobalDecl GD); const CGFunctionInfo &arrangeFreeFunctionCall(const CallArgList &Args, const FunctionType *Ty, - bool ChainCall); + bool ChainCall, + const FunctionDecl *ABIInfoFD); const CGFunctionInfo &arrangeFreeFunctionType(CanQual<FunctionProtoType> Ty); const CGFunctionInfo &arrangeFreeFunctionType(CanQual<FunctionNoProtoType> Ty); @@ -240,9 +252,9 @@ class CodeGenTypes { const CGFunctionInfo &arrangeObjCMethodDeclaration(const ObjCMethodDecl *MD); const CGFunctionInfo &arrangeObjCMessageSendSignature(const ObjCMethodDecl *MD, QualType receiverType); - const CGFunctionInfo &arrangeUnprototypedObjCMessageSend( - QualType returnType, - const CallArgList &args); + const CGFunctionInfo & + arrangeUnprototypedObjCMessageSend(QualType returnType, + const CallArgList &args); /// Block invocation functions are C functions with an implicit parameter. const CGFunctionInfo &arrangeBlockFunctionDeclaration( @@ -254,17 +266,16 @@ class CodeGenTypes { /// C++ methods have some special rules and also have implicit parameters. const CGFunctionInfo &arrangeCXXMethodDeclaration(const CXXMethodDecl *MD); const CGFunctionInfo &arrangeCXXStructorDeclaration(GlobalDecl GD); - const CGFunctionInfo &arrangeCXXConstructorCall(const CallArgList &Args, - const CXXConstructorDecl *D, - CXXCtorType CtorKind, - unsigned ExtraPrefixArgs, - unsigned ExtraSuffixArgs, - bool PassProtoArgs = true); + const CGFunctionInfo &arrangeCXXConstructorCall( + const CallArgList &Args, const CXXConstructorDecl *D, + CXXCtorType CtorKind, unsigned ExtraPrefixArgs, unsigned ExtraSuffixArgs, + const FunctionDecl *ABIInfoFD, bool PassProtoArgs = true); const CGFunctionInfo &arrangeCXXMethodCall(const CallArgList &args, const FunctionProtoType *type, RequiredArgs required, - unsigned numPrefixArgs); + unsigned numPrefixArgs, + const FunctionDecl *ABIInfoFD); const CGFunctionInfo & arrangeUnprototypedMustTailThunk(const CXXMethodDecl *MD); const CGFunctionInfo &arrangeMSCtorClosure(const CXXConstructorDecl *CD, @@ -283,7 +294,7 @@ class CodeGenTypes { CanQualType returnType, FnInfoOpts opts, ArrayRef<CanQualType> argTypes, FunctionType::ExtInfo info, ArrayRef<FunctionProtoType::ExtParameterInfo> paramInfos, - RequiredArgs args); + RequiredArgs args, const FunctionDecl *ABIInfoFD); /// Compute a new LLVM record layout object for the given record. std::unique_ptr<CGRecordLayout> ComputeRecordLayout(const RecordDecl *D, diff --git a/clang/lib/CodeGen/ItaniumCXXABI.cpp b/clang/lib/CodeGen/ItaniumCXXABI.cpp index b17478cb7ffde..5c5fefe32c06c 100644 --- a/clang/lib/CodeGen/ItaniumCXXABI.cpp +++ b/clang/lib/CodeGen/ItaniumCXXABI.cpp @@ -3530,7 +3530,7 @@ ItaniumCXXABI::getOrCreateVirtualFunctionPointerThunk(const CXXMethodDecl *MD) { const FunctionProtoType *FPT = MD->getType()->getAs<FunctionProtoType>(); RequiredArgs Required = RequiredArgs::forPrototypePlus(FPT, /*this*/ 1); const CGFunctionInfo &CallInfo = - CGM.getTypes().arrangeCXXMethodCall(CallArgs, FPT, Required, 0); + CGM.getTypes().arrangeCXXMethodCall(CallArgs, FPT, Required, 0, MD); CGCallee Callee = CGCallee::forVirtual(nullptr, GlobalDecl(MD), getThisAddress(CGF), ThunkTy); llvm::CallBase *CallOrInvoke; diff --git a/clang/lib/CodeGen/MicrosoftCXXABI.cpp b/clang/lib/CodeGen/MicrosoftCXXABI.cpp index d324580036f6a..8b43dd887573d 100644 --- a/clang/lib/CodeGen/MicrosoftCXXABI.cpp +++ b/clang/lib/CodeGen/MicrosoftCXXABI.cpp @@ -4229,8 +4229,12 @@ MicrosoftCXXABI::getAddrOfCXXCtorClosure(const CXXConstructorDecl *CD, CGM.getAddrOfCXXStructor(GlobalDecl(CD, Ctor_Complete)); CGCallee Callee = CGCallee::forDirect(CalleePtr, GlobalDecl(CD, Ctor_Complete)); + // Microsoft ABI constructors always use the default method calling + // convention (see SemaType.cpp adjustMemberFunctionCC), so no caller + // declaration is needed for SysV ABI selection. const CGFunctionInfo &CalleeInfo = CGM.getTypes().arrangeCXXConstructorCall( - Args, CD, Ctor_Complete, ExtraArgs.Prefix, ExtraArgs.Suffix); + Args, CD, Ctor_Complete, ExtraArgs.Prefix, ExtraArgs.Suffix, + /*ABIInfoFD=*/nullptr); CGF.EmitCall(CalleeInfo, Callee, ReturnValueSlot(), Args); Cleanups.ForceCleanup(); diff --git a/clang/lib/CodeGen/Targets/X86.cpp b/clang/lib/CodeGen/Targets/X86.cpp index 807d372c3c8fd..7c71ed9b34e83 100644 --- a/clang/lib/CodeGen/Targets/X86.cpp +++ b/clang/lib/CodeGen/Targets/X86.cpp @@ -1385,6 +1385,8 @@ class X86_64ABIInfo : public ABIInfo { } void computeInfo(CGFunctionInfo &FI) const override; + unsigned getX86ABIAVXLevel(const FunctionDecl *FD, + const FunctionType::ExtInfo &Info) const override; RValue EmitVAArg(CodeGenFunction &CGF, Address VAListAddr, QualType Ty, AggValueSlot Slot) const override; @@ -1404,6 +1406,8 @@ class WinX86_64ABIInfo : public ABIInfo { IsMingw64(getTarget().getTriple().isWindowsGNUEnvironment()) {} void computeInfo(CGFunctionInfo &FI) const override; + unsigned getX86ABIAVXLevel(const FunctionDecl *FD, + const FunctionType::ExtInfo &Info) const override; RValue EmitVAArg(CodeGenFunction &CGF, Address VAListAddr, QualType Ty, AggValueSlot Slot) const override; @@ -1822,6 +1826,29 @@ void X86_64ABIInfo::postMerge(unsigned AggregateSize, Class &Lo, Hi = SSE; } +static X86AVXABILevel getEffectiveX86AVXABILevel(CodeGenTypes &CGT, + X86AVXABILevel GlobalAVXLevel, + const FunctionDecl *FD) { + // Always return global AVX level on PlayStation. + if (CGT.getTarget().getTriple().isPS() || + CGT.getContext().getLangOpts().getClangABICompat() <= + LangOptions::ClangABI::Ver23) { + return GlobalAVXLevel; + } + + X86AVXABILevel Level = GlobalAVXLevel; + if (!FD) + return Level; + + llvm::StringMap<bool> FeatureMap; + CGT.getCGM().getContext().getFunctionFeatureMap(FeatureMap, FD); + if (FeatureMap.lookup("avx512f")) + return std::max(Level, X86AVXABILevel::AVX512); + if (FeatureMap.lookup("avx")) + return std::max(Level, X86AVXABILevel::AVX); + return Level; +} + X86_64ABIInfo::Class X86_64ABIInfo::merge(Class Accum, Class Field) { // AMD64-ABI 3.2.3p2: Rule 4. Each field of an object is // classified recursively so that always two fields are @@ -3024,8 +3051,13 @@ X86_64ABIInfo::classifyRegCallStructType(QualType Ty, unsigned &NeededInt, llvm::StructType::get(getVMContext(), CoerceElts)); } -void X86_64ABIInfo::computeInfo(CGFunctionInfo &FI) const { +unsigned +X86_64ABIInfo::getX86ABIAVXLevel(const FunctionDecl *FD, + const FunctionType::ExtInfo &Info) const { + return static_cast<unsigned>(getEffectiveX86AVXABILevel(CGT, AVXLevel, FD)); +} +void X86_64ABIInfo::computeInfo(CGFunctionInfo &FI) const { const unsigned CallingConv = FI.getCallingConvention(); // It is possible to force Win64 calling convention on any x86_64 target by // using __attribute__((ms_abi)). In such case to correctly emit Win64 @@ -3036,6 +3068,17 @@ void X86_64ABIInfo::computeInfo(CGFunctionInfo &FI) const { return; } + assert(FI.getX86ABIAVXLevel() <= + static_cast<unsigned>(X86AVXABILevel::AVX512) && + "Unexpected X86 AVX ABI level"); + X86AVXABILevel EffectiveAVXLevel = + static_cast<X86AVXABILevel>(FI.getX86ABIAVXLevel()); + if (EffectiveAVXLevel != AVXLevel) { + X86_64ABIInfo EffectiveABIInfo(CGT, EffectiveAVXLevel); + EffectiveABIInfo.computeInfo(FI); + return; + } + bool IsRegCall = CallingConv == llvm::CallingConv::X86_RegCall; // Keep track of the number of assigned registers. @@ -3558,6 +3601,16 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs, return ABIArgInfo::getDirect(); } +unsigned +WinX86_64ABIInfo::getX86ABIAVXLevel(const FunctionDecl *FD, + const FunctionType::ExtInfo &Info) const { + if (Info.getCC() == CC_X86_64SysV) { + return static_cast<unsigned>(getEffectiveX86AVXABILevel(CGT, AVXLevel, FD)); + } + + return static_cast<unsigned>(AVXLevel); +} + void WinX86_64ABIInfo::computeInfo(CGFunctionInfo &FI) const { const unsigned CC = FI.getCallingConvention(); bool IsVectorCall = CC == llvm::CallingConv::X86_VectorCall; diff --git a/clang/test/CodeGen/sysv_abi.c b/clang/test/CodeGen/sysv_abi.c index a66ecc6e26242..81d2b26a09629 100644 --- a/clang/test/CodeGen/sysv_abi.c +++ b/clang/test/CodeGen/sysv_abi.c @@ -53,3 +53,21 @@ void use_vectors(void) { // NOAVX: call {{(x86_64_sysvcc )?}}void @take_m256(ptr noundef byval(<8 x float>) align 32 %{{.*}}) // NOAVX: call {{(x86_64_sysvcc )?}}<16 x float> @get_m512() // NOAVX: call {{(x86_64_sysvcc )?}}void @take_m512(ptr noundef byval(<16 x float>) align 64 %{{.*}}) + +// Added test to explicitly cover the case when __attribute__((target("avx"))) is used +// with __attribute__((sysv_abi)) + +__attribute__((target("avx"))) my_m256 SYSV_CC get_avx_m256(void); +__attribute__((target("avx"))) void SYSV_CC take_avx_m256(my_m256); + +__attribute__((target("avx"))) +void use_target_attr_vectors(void) { + my_m256 v = get_avx_m256(); + take_avx_m256(v); +} + +// CHECK: define {{(dso_local )?}}void @use_target_attr_vectors() +// AVX: call {{(x86_64_sysvcc )?}}<8 x float> @get_avx_m256() +// AVX: call {{(x86_64_sysvcc )?}}void @take_avx_m256(<8 x float> noundef %{{.*}}) +// NOAVX: call {{(x86_64_sysvcc )?}}<8 x float> @get_avx_m256() +// NOAVX: call {{(x86_64_sysvcc )?}}void @take_avx_m256(<8 x float> noundef %{{.*}}) diff --git a/clang/test/CodeGen/target-avx-function-abi.c b/clang/test/CodeGen/target-avx-function-abi.c new file mode 100644 index 0000000000000..6ea0fd39a29f1 --- /dev/null +++ b/clang/test/CodeGen/target-avx-function-abi.c @@ -0,0 +1,71 @@ +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=SYSV +// RUN: %clang_cc1 -triple x86_64-scei-ps4 -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=PS +// RUN: %clang_cc1 -triple x86_64-sie-ps5 -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=PS + +typedef float v8f __attribute__((vector_size(32))); +typedef float v16f __attribute__((vector_size(64))); + +v8f g256(v8f x) { return x; } +v16f g512(v16f x) { return x; } + +__attribute__((target("avx"))) v8f l256(v8f x) { return x; } +__attribute__((target("avx512f"))) v16f l512(v16f x) { return x; } + +__attribute__((target("avx"))) v8f call_l256(v8f x) { return l256(x); } +__attribute__((target("avx512f"))) v16f call_l512(v16f x) { return l512(x); } + +__attribute__((target("avx"))) v8f call_ptr_l256(v8f x) { + v8f (*fp)(v8f) = l256; + return fp(x); +} + +__attribute__((target("avx512f"))) v16f call_ptr_l512(v16f x) { + v16f (*fp)(v16f) = l512; + return fp(x); +} + +// SYSV-LABEL: define dso_local <8 x float> @g256( +// SYSV: byval(<8 x float>) align 32 + +// SYSV-LABEL: define dso_local <16 x float> @g512( +// SYSV: byval(<16 x float>) align 64 + +// SYSV-LABEL: define dso_local <8 x float> @l256(<8 x float> noundef %x) +// SYSV-LABEL: define dso_local <16 x float> @l512(<16 x float> noundef %x) + +// SYSV-LABEL: define dso_local <8 x float> @call_l256(<8 x float> noundef %x) +// SYSV: call <8 x float> @l256(<8 x float> noundef + +// SYSV-LABEL: define dso_local <16 x float> @call_l512(<16 x float> noundef %x) +// SYSV: call <16 x float> @l512(<16 x float> noundef + +// SYSV-LABEL: define dso_local <8 x float> @call_ptr_l256(<8 x float> noundef %x) +// SYSV: call <8 x float> %{{.*}}(<8 x float> noundef + +// SYSV-LABEL: define dso_local <16 x float> @call_ptr_l512(<16 x float> noundef %x) +// SYSV: call <16 x float> %{{.*}}(<16 x float> noundef + +// PlayStation keeps the legacy ABI which always returns AVX vectors in registers & only uses AVX level from module/TU level even with AVX target attributes. +// PS-LABEL: define dso_local <8 x float> @g256( +// PS: byval(<8 x float>) align 32 + +// PS-LABEL: define dso_local <16 x float> @g512( +// PS: byval(<16 x float>) align 64 + +// PS-LABEL: define dso_local <8 x float> @l256( +// PS: byval(<8 x float>) align 32 + +// PS-LABEL: define dso_local <16 x float> @l512( +// PS: byval(<16 x float>) align 64 + +// PS-LABEL: define dso_local <8 x float> @call_l256( +// PS: call <8 x float> @l256(ptr noundef byval(<8 x float>) align 32 + +// PS-LABEL: define dso_local <16 x float> @call_l512( +// PS: call <16 x float> @l512(ptr noundef byval(<16 x float>) align 64 + +// PS-LABEL: define dso_local <8 x float> @call_ptr_l256( +// PS: call <8 x float> %{{.*}}(ptr noundef byval(<8 x float>) align 32 + +// PS-LABEL: define dso_local <16 x float> @call_ptr_l512( +// PS: call <16 x float> %{{.*}}(ptr noundef byval(<16 x float>) align 64 diff --git a/clang/test/CodeGenCXX/target-avx-method-abi.cpp b/clang/test/CodeGenCXX/target-avx-method-abi.cpp new file mode 100644 index 0000000000000..47bd1d3ca8c83 --- /dev/null +++ b/clang/test/CodeGenCXX/target-avx-method-abi.cpp @@ -0,0 +1,121 @@ +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -std=c++20 -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=SYSV +// RUN: %clang_cc1 -triple x86_64-scei-ps4 -std=c++20 -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=PS +// RUN: %clang_cc1 -triple x86_64-sie-ps5 -std=c++20 -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=PS + +typedef float v8f __attribute__((vector_size(32))); +typedef float v16f __attribute__((vector_size(64))); + +struct S { + __attribute__((target("avx"))) + v8f m(v8f x) { return x; } +}; + +struct C { + v8f field; + __attribute__((target("avx"))) + C(v8f x) : field(x) {} +}; + +__attribute__((target("avx"))) +v8f callm(S *s, v8f x) { + return s->m(x); +} + +__attribute__((target("avx"))) +v8f callctor(v8f x) { + C c(x); + return c.field; +} + +__attribute__((target("avx"))) +v8f callm_ptr(S *s, v8f x) { + v8f (S::*pmf)(v8f) = &S::m; + return (s->*pmf)(x); +} + +struct S512 { + __attribute__((target("avx512f"))) + v16f m512(v16f x) { return x; } +}; + +struct D512 { + v16f field; + __attribute__((target("avx512f"))) + D512(v16f x) : field(x) {} +}; + +__attribute__((target("avx512f"))) +v16f callm512(S512 *s, v16f x) { + return s->m512(x); +} + +__attribute__((target("avx512f"))) +v16f callctor512(v16f x) { + D512 c(x); + return c.field; +} + +__attribute__((target("avx512f"))) +v16f callm512_ptr(S512 *s, v16f x) { + v16f (S512::*pmf)(v16f) = &S512::m512; + return (s->*pmf)(x); +} + +// Desired ABI behavior: AVX-targeted member functions should pass/return AVX +// vectors directly, just like AVX-targeted free functions. +// SYSV-LABEL: define dso_local noundef <8 x float> @_Z5callmP1SDv8_f( +// SYSV: call noundef <8 x float> @_ZN1S1mEDv8_f(ptr noundef +// SYSV-LABEL: define linkonce_odr noundef <8 x float> @_ZN1S1mEDv8_f( +// SYSV-SAME: ptr noundef nonnull align 1 dereferenceable(1) %this, <8 x float> noundef %x) +// SYSV-LABEL: define dso_local noundef <8 x float> @_Z8callctorDv8_f( +// SYSV: call void @_ZN1CC1EDv8_f(ptr noundef nonnull align 32 dereferenceable(32) +// SYSV-SAME: <8 x float> noundef +// SYSV-LABEL: define linkonce_odr void @_ZN1CC1EDv8_f( +// SYSV-SAME: ptr noundef nonnull align 32 dereferenceable(32) %this, <8 x float> noundef %x) +// SYSV-LABEL: define dso_local noundef <8 x float> @_Z9callm_ptrP1SDv8_f( +// SYSV-SAME: ptr noundef %s, <8 x float> noundef %x) +// SYSV: call noundef <8 x float> %{{.*}}(ptr noundef nonnull align 1 dereferenceable(1) %{{.*}}, <8 x float> noundef +// SYSV-LABEL: define dso_local noundef <16 x float> @_Z8callm512P4S512Dv16_f( +// SYSV: call noundef <16 x float> @_ZN4S5124m512EDv16_f(ptr noundef +// SYSV-LABEL: define linkonce_odr noundef <16 x float> @_ZN4S5124m512EDv16_f( +// SYSV-SAME: ptr noundef nonnull align 1 dereferenceable(1) %this, <16 x float> noundef %x) +// SYSV-LABEL: define dso_local noundef <16 x float> @_Z11callctor512Dv16_f( +// SYSV: call void @_ZN4D512C1EDv16_f(ptr noundef nonnull align 64 dereferenceable(64) +// SYSV-SAME: <16 x float> noundef +// SYSV-LABEL: define linkonce_odr void @_ZN4D512C1EDv16_f( +// SYSV-SAME: ptr noundef nonnull align 64 dereferenceable(64) %this, <16 x float> noundef %x) +// SYSV-LABEL: define dso_local noundef <16 x float> @_Z12callm512_ptrP4S512Dv16_f( +// SYSV-SAME: ptr noundef %s, <16 x float> noundef %x) +// SYSV: call noundef <16 x float> %{{.*}}(ptr noundef nonnull align 1 dereferenceable(1) %{{.*}}, <16 x float> noundef +// SYSV-LABEL: define linkonce_odr void @_ZN1CC2EDv8_f( +// SYSV-SAME: ptr noundef nonnull align 32 dereferenceable(32) %this, <8 x float> noundef %x) +// SYSV-LABEL: define linkonce_odr void @_ZN4D512C2EDv16_f( +// SYSV-SAME: ptr noundef nonnull align 64 dereferenceable(64) %this, <16 x float> noundef %x) + +// PlayStation keeps the legacy ABI which always returns AVX vectors in registers & only uses AVX level from module/TU level even with AVX target attributes. +// PS-LABEL: define dso_local noundef <8 x float> @_Z5callmP1SDv8_f( +// PS: byval(<8 x float>) align 32 +// PS: call noundef <8 x float> @_ZN1S1mEDv8_f( +// PS-LABEL: define linkonce_odr noundef <8 x float> @_ZN1S1mEDv8_f( +// PS: byval(<8 x float>) align 32 +// PS-LABEL: define dso_local noundef <8 x float> @_Z8callctorDv8_f( +// PS-LABEL: define linkonce_odr void @_ZN1CC1EDv8_f( +// PS-SAME: ptr noundef nonnull align 32 dereferenceable(32) %this, ptr noundef byval(<8 x float>) align 32 +// PS-LABEL: define dso_local noundef <8 x float> @_Z9callm_ptrP1SDv8_f( +// PS-SAME: ptr noundef %s, ptr noundef byval(<8 x float>) align 32 +// PS: call noundef <8 x float> %{{.*}}(ptr noundef nonnull align 1 dereferenceable(1) %{{.*}}, ptr noundef byval(<8 x float>) align 32 +// PS-LABEL: define dso_local noundef <16 x float> @_Z8callm512P4S512Dv16_f( +// PS: byval(<16 x float>) align 64 +// PS: call noundef <16 x float> @_ZN4S5124m512EDv16_f( +// PS-LABEL: define linkonce_odr noundef <16 x float> @_ZN4S5124m512EDv16_f( +// PS: byval(<16 x float>) align 64 +// PS-LABEL: define dso_local noundef <16 x float> @_Z11callctor512Dv16_f( +// PS-LABEL: define linkonce_odr void @_ZN4D512C1EDv16_f( +// PS-SAME: ptr noundef nonnull align 64 dereferenceable(64) %this, ptr noundef byval(<16 x float>) align 64 +// PS-LABEL: define dso_local noundef <16 x float> @_Z12callm512_ptrP4S512Dv16_f( +// PS-SAME: ptr noundef %s, ptr noundef byval(<16 x float>) align 64 +// PS: call noundef <16 x float> %{{.*}}(ptr noundef nonnull align 1 dereferenceable(1) %{{.*}}, ptr noundef byval(<16 x float>) align 64 +// PS-LABEL: define linkonce_odr void @_ZN1CC2EDv8_f( +// PS-SAME: ptr noundef nonnull align 32 dereferenceable(32) %this, ptr noundef byval(<8 x float>) align 32 +// PS-LABEL: define linkonce_odr void @_ZN4D512C2EDv16_f( +// PS-SAME: ptr noundef nonnull align 64 dereferenceable(64) %this, ptr noundef byval(<16 x float>) align 64 diff --git a/clang/unittests/CodeGen/CodeGenExternalTest.cpp b/clang/unittests/CodeGen/CodeGenExternalTest.cpp index 8824451ccc2f4..b98f6c513548f 100644 --- a/clang/unittests/CodeGen/CodeGenExternalTest.cpp +++ b/clang/unittests/CodeGen/CodeGenExternalTest.cpp @@ -11,6 +11,7 @@ #include "clang/AST/ASTConsumer.h" #include "clang/AST/ASTContext.h" #include "clang/AST/BaseSubobject.h" +#include "clang/AST/Decl.h" #include "clang/AST/GlobalDecl.h" #include "clang/AST/RecursiveASTVisitor.h" #include "clang/Basic/TargetInfo.h" @@ -44,8 +45,10 @@ static const bool DebugThisTest = false; // forward declarations struct MyASTConsumer; -static void test_codegen_fns(MyASTConsumer *my); -static bool test_codegen_fns_ran; +static void test_generic_codegen_fns(MyASTConsumer *my); +static void test_x86_avx_abi_codegen_fns(MyASTConsumer *my); +static bool test_generic_codegen_fns_ran; +static bool test_x86_avx_abi_codegen_fns_ran; // This forwards the calls to the Clang CodeGenerator // so that we can test CodeGen functions while it is open. @@ -54,13 +57,15 @@ static bool test_codegen_fns_ran; // before forwarding that function to the CodeGenerator. struct MyASTConsumer : public ASTConsumer { + using TestFn = void (*)(MyASTConsumer *); + std::unique_ptr<CodeGenerator> Builder; + TestFn TestFunction; std::vector<Decl*> toplevel_decls; - MyASTConsumer(std::unique_ptr<CodeGenerator> Builder_in) - : ASTConsumer(), Builder(std::move(Builder_in)) - { - } + MyASTConsumer(std::unique_ptr<CodeGenerator> Builder_in, TestFn TestFunction) + : ASTConsumer(), Builder(std::move(Builder_in)), + TestFunction(TestFunction) {} ~MyASTConsumer() { } @@ -106,7 +111,7 @@ void MyASTConsumer::HandleInterestingDecl(DeclGroupRef D) { } void MyASTConsumer::HandleTranslationUnit(ASTContext &Context) { - test_codegen_fns(this); + TestFunction(this); // HandleTranslationUnit can close the module Builder->HandleTranslationUnit(Context); } @@ -163,7 +168,7 @@ bool MyASTConsumer::shouldSkipFunctionBody(Decl *D) { return Builder->shouldSkipFunctionBody(D); } -const char TestProgram[] = +const char GenericTestProgram[] = "struct mytest_struct { char x; short y; char p; long z; };\n" "int mytest_fn(int x) { return x; }\n" "struct mytest_dynamic_struct {\n" @@ -172,9 +177,27 @@ const char TestProgram[] = "};\n" "mytest_dynamic_struct::mytest_dynamic_struct() { }\n"; -// This function has the real test code here -static void test_codegen_fns(MyASTConsumer *my) { - +const char X86AVXABITestProgram[] = + "typedef float mytest_v8f __attribute__((vector_size(32)));\n" + "struct mytest_avx_method_holder {\n" + " __attribute__((target(\"avx\"))) mytest_v8f method(mytest_v8f x) {\n" + " return x;\n" + " }\n" + "};\n" + "__attribute__((target(\"avx\"))) mytest_v8f mytest_avx_fn(mytest_v8f x) " + "{\n" + " return x;\n" + "}\n" + "__attribute__((target(\"avx\"))) void caller() " + "{\n" + " mytest_v8f hello = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};\n" + " mytest_avx_fn(hello);\n" + " mytest_avx_method_holder holder = mytest_avx_method_holder();\n" + " holder.method(hello);\n" + "}\n" + "__attribute__((target(\"avx\"))) void caller2() {}\n"; + +static void test_generic_codegen_fns(MyASTConsumer *my) { bool mytest_fn_ok = false; bool mytest_struct_ok = false; @@ -281,7 +304,81 @@ static void test_codegen_fns(MyASTConsumer *my) { ASSERT_TRUE(mytest_fn_ok); ASSERT_TRUE(mytest_struct_ok); - test_codegen_fns_ran = true; + test_generic_codegen_fns_ran = true; +} + +static void test_x86_avx_abi_codegen_fns(MyASTConsumer *my) { + bool mytest_avx_fn_ok = false; + bool mytest_avx_method_ok = false; + + CodeGen::CodeGenModule &CGM = my->Builder->CGM(); + const ASTContext &Ctx = my->toplevel_decls.front()->getASTContext(); + + // First get the caller decls, which we need to determine call ABI. + FunctionDecl *callerDecl = nullptr; + FunctionDecl *caller2Decl = nullptr; + for (auto decl : my->toplevel_decls) { + if (FunctionDecl *fd = dyn_cast<FunctionDecl>(decl)) { + if (fd->getName() == "caller") + callerDecl = fd; + else if (fd->getName() == "caller2") + caller2Decl = fd; + } + } + ASSERT_TRUE(callerDecl); + ASSERT_TRUE(caller2Decl); + ASSERT_NE(callerDecl, caller2Decl); + + for (auto decl : my->toplevel_decls) { + if (FunctionDecl *fd = dyn_cast<FunctionDecl>(decl)) { + if (fd->getName() != "mytest_avx_fn") + continue; + + const auto *FPT = fd->getType()->castAs<FunctionProtoType>(); + SmallVector<CanQualType, 4> ArgTypes; + for (const ParmVarDecl *Param : fd->parameters()) + ArgTypes.push_back(Ctx.getCanonicalParamType(Param->getType())); + + const CodeGen::CGFunctionInfo &FnInfo = CodeGen::arrangeFreeFunctionCall( + CGM, Ctx.getCanonicalType(FPT->getReturnType()), ArgTypes, + FPT->getExtInfo(), {}, + CodeGen::RequiredArgs::forPrototypePlus(FPT, 0), callerDecl); + const CodeGen::CGFunctionInfo &FnInfo2 = CodeGen::arrangeFreeFunctionCall( + CGM, Ctx.getCanonicalType(FPT->getReturnType()), ArgTypes, + FPT->getExtInfo(), {}, + CodeGen::RequiredArgs::forPrototypePlus(FPT, 0), caller2Decl); + ASSERT_EQ(&FnInfo, &FnInfo2); + ASSERT_EQ(FnInfo.getX86ABIAVXLevel(), 1u); + ASSERT_TRUE(FnInfo.getReturnInfo().isDirect()); + ASSERT_TRUE(FnInfo.arg_begin()->info.isDirect()); + mytest_avx_fn_ok = true; + } else if (RecordDecl *rd = dyn_cast<RecordDecl>(decl)) { + if (rd->getName() != "mytest_avx_method_holder") + continue; + + const auto *MethodRD = cast<CXXRecordDecl>(rd->getDefinition()); + const auto *MD = cast<CXXMethodDecl>(*MethodRD->method_begin()); + const auto *FPT = MD->getType()->castAs<FunctionProtoType>(); + SmallVector<CanQualType, 4> ArgTypes; + ArgTypes.push_back(Ctx.getCanonicalParamType(MD->getThisType())); + for (const ParmVarDecl *Param : MD->parameters()) + ArgTypes.push_back(Ctx.getCanonicalParamType(Param->getType())); + + const CodeGen::CGFunctionInfo &FnInfo = CodeGen::arrangeCXXMethodCall( + CGM, Ctx.getCanonicalType(FPT->getReturnType()), ArgTypes, + FPT->getExtInfo(), {}, + CodeGen::RequiredArgs::forPrototypePlus(FPT, 1), callerDecl); + ASSERT_EQ(FnInfo.getX86ABIAVXLevel(), 1u); + ASSERT_TRUE(FnInfo.getReturnInfo().isDirect()); + ASSERT_TRUE(FnInfo.arg_begin()[1].info.isDirect()); + mytest_avx_method_ok = true; + } + } + + ASSERT_TRUE(mytest_avx_fn_ok); + ASSERT_TRUE(mytest_avx_method_ok); + + test_x86_avx_abi_codegen_fns_ran = true; } TEST(CodeGenExternalTest, CodeGenExternalTest) { @@ -289,14 +386,30 @@ TEST(CodeGenExternalTest, CodeGenExternalTest) { LO.CPlusPlus = 1; LO.CPlusPlus11 = 1; TestCompiler Compiler(LO); - auto CustomASTConsumer - = std::make_unique<MyASTConsumer>(std::move(Compiler.CG)); + auto CustomASTConsumer = std::make_unique<MyASTConsumer>( + std::move(Compiler.CG), test_generic_codegen_fns); + + Compiler.init(GenericTestProgram, std::move(CustomASTConsumer)); + + clang::ParseAST(Compiler.compiler.getSema(), false, false); + + ASSERT_TRUE(test_generic_codegen_fns_ran); +} + +TEST(CodeGenExternalTest, X86AVXABIQuery) { + clang::LangOptions LO; + LO.CPlusPlus = 1; + LO.CPlusPlus11 = 1; + TestCompiler Compiler(LO, clang::CodeGenOptions(), + "x86_64-unknown-linux-gnu"); + auto CustomASTConsumer = std::make_unique<MyASTConsumer>( + std::move(Compiler.CG), test_x86_avx_abi_codegen_fns); - Compiler.init(TestProgram, std::move(CustomASTConsumer)); + Compiler.init(X86AVXABITestProgram, std::move(CustomASTConsumer)); clang::ParseAST(Compiler.compiler.getSema(), false, false); - ASSERT_TRUE(test_codegen_fns_ran); + ASSERT_TRUE(test_x86_avx_abi_codegen_fns_ran); } } // end anonymous namespace diff --git a/clang/unittests/CodeGen/TestCompiler.h b/clang/unittests/CodeGen/TestCompiler.h index 18947584bd0b3..3ba839979b867 100644 --- a/clang/unittests/CodeGen/TestCompiler.h +++ b/clang/unittests/CodeGen/TestCompiler.h @@ -33,17 +33,21 @@ struct TestCompiler { unsigned PtrSize = 0; TestCompiler(clang::LangOptions LO, - clang::CodeGenOptions CGO = clang::CodeGenOptions()) { + clang::CodeGenOptions CGO = clang::CodeGenOptions(), + llvm::StringRef TripleStr = "") { compiler.getLangOpts() = LO; compiler.getCodeGenOpts() = CGO; compiler.setVirtualFileSystem(llvm::vfs::getRealFileSystem()); compiler.createDiagnostics(); - std::string TrStr = llvm::Triple::normalize(llvm::sys::getProcessTriple()); - llvm::Triple Tr(TrStr); - Tr.setOS(Triple::Linux); - Tr.setVendor(Triple::VendorType::UnknownVendor); - Tr.setEnvironment(Triple::EnvironmentType::UnknownEnvironment); + llvm::Triple Tr(TripleStr.empty() + ? llvm::Triple::normalize(llvm::sys::getProcessTriple()) + : llvm::Triple::normalize(TripleStr)); + if (TripleStr.empty()) { + Tr.setOS(Triple::Linux); + Tr.setVendor(Triple::VendorType::UnknownVendor); + Tr.setEnvironment(Triple::EnvironmentType::UnknownEnvironment); + } compiler.getTargetOpts().Triple = Tr.getTriple(); compiler.setTarget(clang::TargetInfo::CreateTargetInfo( compiler.getDiagnostics(), compiler.getTargetOpts())); >From 3548b49f00de523193be2239de4f3491e5998aac Mon Sep 17 00:00:00 2001 From: Benjamin Luke <[email protected]> Date: Fri, 17 Jul 2026 07:27:37 -0700 Subject: [PATCH 2/2] Only call getFunctionFeatureMap if the FD has target attributes to avoid compile time regressions --- clang/lib/CodeGen/Targets/X86.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/CodeGen/Targets/X86.cpp b/clang/lib/CodeGen/Targets/X86.cpp index 7c71ed9b34e83..bf912f643a11b 100644 --- a/clang/lib/CodeGen/Targets/X86.cpp +++ b/clang/lib/CodeGen/Targets/X86.cpp @@ -1837,7 +1837,7 @@ static X86AVXABILevel getEffectiveX86AVXABILevel(CodeGenTypes &CGT, } X86AVXABILevel Level = GlobalAVXLevel; - if (!FD) + if (!FD || !FD->hasAttr<TargetAttr>()) return Level; llvm::StringMap<bool> FeatureMap; _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
