samitolvanen created this revision.
Herald added subscribers: dexonsmith, dang, jdoerfert, cryptoad.
Herald added a reviewer: aaron.ballman.
samitolvanen requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
The KCFI sanitizer, enabled with -fsanitize=kcfi, implements a
forward-edge control flow integrity scheme in Clang for indirect
calls. It uses LLVM prefix data to store a type identifier for each
function and injects verification code before indirect calls.
Unlike the current CFI schemes implemented in LLVM, KCFI does not
require LTO, does not alter function references to point to a jump
table, and never breaks function address equality. KCFI is intended
to be used in low-level code, such as operating system kernels,
where the existing schemes can cause undue complications because
of the aforementioned properties. However, unlike the existing
schemes, KCFI is limited to validating only indirect calls and is
not compatible with executable-only memory.
KCFI does not provide runtime support. Users of the scheme are
expected to implement an error handling function with the following
signature:
void __kcfi_check_fail(uint64_t id, void *ptr);
The first argument is the expected type identifier, and the second
argument is the address of the indirect call. This function must not
return unless -fsanitize-recover=kcfi is passed to Clang, in which
case code execution may continue after the error handler.
A KCFI type identifier is the xxHash64 of a C++ mangled typename,
with the first two bytes replaced with 0x10CF to make it easier for
binary tooling to detect the identifiers in programs where not all
functions have one. If a program contains indirect calls to assembly
functions, they must be manually annotated with the expected type
identifiers to prevent errors. To make this easier, Clang generates
a __kcfi_typeid_ symbol for each address-taken function declaration,
which can be used to annotate functions in assembly:
.c:
int f(void);
int (*p)(void) = f;
p();
.s:
.quad __kcfi_typeid_f
.global f
f:
...
Note that if additional data has been injected between the KCFI
type identifier and the start of the function, e.g. by using
-fpatchable-function-entry, the offset in bytes must be specified
using -fsanitize-kcfi-offset=<value> to avoid errors. The offset
must be the same for all indirectly called functions in every
translation unit.
Similarly to other sanitizers, KCFI checking can be disabled for
a function with a no_sanitize("kcfi") function attribute, or by
using the sanitizer special case list. However, as this may not
be fine-grained enough in cases where want to disable checking
only for specific calls, we also add the kcfi_unchecked attribute
for function pointers and function pointer types, which disables
checking for indirect calls made using an annotated pointer.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D119296
Files:
clang/docs/ClangCommandLineReference.rst
clang/docs/UsersManual.rst
clang/include/clang/Basic/Attr.td
clang/include/clang/Basic/AttrDocs.td
clang/include/clang/Basic/CodeGenOptions.def
clang/include/clang/Basic/DiagnosticSemaKinds.td
clang/include/clang/Basic/Features.def
clang/include/clang/Basic/Sanitizers.def
clang/include/clang/Driver/Options.td
clang/include/clang/Driver/SanitizerArgs.h
clang/lib/CodeGen/CGExpr.cpp
clang/lib/CodeGen/CodeGenFunction.h
clang/lib/CodeGen/CodeGenModule.cpp
clang/lib/CodeGen/CodeGenModule.h
clang/lib/Driver/SanitizerArgs.cpp
clang/lib/Driver/ToolChain.cpp
clang/lib/Sema/SemaDeclAttr.cpp
clang/test/CodeGen/kcfi.c
clang/test/CodeGen/kcfi_unchecked.c
clang/test/Driver/fsanitize.c
clang/test/Misc/pragma-attribute-supported-attributes-list.test
clang/test/Sema/attr-kcfi_unchecked.c
Index: clang/test/Sema/attr-kcfi_unchecked.c
===================================================================
--- /dev/null
+++ clang/test/Sema/attr-kcfi_unchecked.c
@@ -0,0 +1,12 @@
+// RUN: %clang_cc1 %s -verify -fsyntax-only
+
+int a __attribute__((kcfi_unchecked)); // expected-error {{'kcfi_unchecked' attribute argument only applies to a function pointer or a function pointer type}}
+void *p __attribute__((kcfi_unchecked)); // expected-error {{'kcfi_unchecked' attribute argument only applies to a function pointer or a function pointer type}}
+void (*f)(void) __attribute__((kcfi_unchecked));
+
+typedef unsigned long l_unchecked_t __attribute__((kcfi_unchecked)); // expected-error {{'kcfi_unchecked' attribute argument only applies to a function pointer or a function pointer type}}
+typedef int (*f_unchecked_t)(void) __attribute__((kcfi_unchecked));
+
+void f1(unsigned long p __attribute__((kcfi_unchecked))) {} // expected-error {{'kcfi_unchecked' attribute argument only applies to a function pointer or a function pointer type}}
+void f2(void *p __attribute__((kcfi_unchecked))) {} // expected-error {{'kcfi_unchecked' attribute argument only applies to a function pointer or a function pointer type}}
+void f3(void (*p)(void) __attribute__((kcfi_unchecked))) {}
Index: clang/test/Misc/pragma-attribute-supported-attributes-list.test
===================================================================
--- clang/test/Misc/pragma-attribute-supported-attributes-list.test
+++ clang/test/Misc/pragma-attribute-supported-attributes-list.test
@@ -76,6 +76,7 @@
// CHECK-NEXT: IFunc (SubjectMatchRule_function)
// CHECK-NEXT: InitPriority (SubjectMatchRule_variable)
// CHECK-NEXT: InternalLinkage (SubjectMatchRule_variable, SubjectMatchRule_function, SubjectMatchRule_record)
+// CHECK-NEXT: KCFIUnchecked (SubjectMatchRule_variable, SubjectMatchRule_type_alias)
// CHECK-NEXT: LTOVisibilityPublic (SubjectMatchRule_record)
// CHECK-NEXT: Leaf (SubjectMatchRule_function)
// CHECK-NEXT: LoaderUninitialized (SubjectMatchRule_variable_is_global)
Index: clang/test/Driver/fsanitize.c
===================================================================
--- clang/test/Driver/fsanitize.c
+++ clang/test/Driver/fsanitize.c
@@ -667,6 +667,19 @@
// RUN: %clang -target x86_64-linux-gnu -fsanitize=cfi -fsanitize-stats -flto -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CFI-STATS
// CHECK-CFI-STATS: -fsanitize-stats
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=kcfi -fsanitize=cfi -flto -fvisibility=hidden %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-KCFI-NOCFI
+// CHECK-KCFI-NOCFI: error: invalid argument '-fsanitize=cfi*' not allowed with '-fsanitize=kcfi'
+
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=kcfi -fsanitize-trap=kcfi %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-KCFI-NOTRAP
+// CHECK-KCFI-NOTRAP: error: unsupported argument 'kcfi' to option '-fsanitize-trap'
+
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=kcfi %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-KCFI
+// CHECK-KCFI: "-fsanitize=kcfi"
+
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=kcfi -fsanitize-recover=kcfi %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-KCFI-RECOVER
+// CHECK-KCFI-RECOVER: "-fsanitize=kcfi"
+// CHECK-KCFI-RECOVER: "-fsanitize-recover=kcfi"
+
// RUN: %clang_cl -fsanitize=address -c -MDd -### -- %s 2>&1 | FileCheck %s -check-prefix=CHECK-ASAN-DEBUGRTL
// RUN: %clang_cl -fsanitize=address -c -MTd -### -- %s 2>&1 | FileCheck %s -check-prefix=CHECK-ASAN-DEBUGRTL
// RUN: %clang_cl -fsanitize=address -c -LDd -### -- %s 2>&1 | FileCheck %s -check-prefix=CHECK-ASAN-DEBUGRTL
Index: clang/test/CodeGen/kcfi_unchecked.c
===================================================================
--- /dev/null
+++ clang/test/CodeGen/kcfi_unchecked.c
@@ -0,0 +1,120 @@
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm -fsanitize=kcfi -o - %s | FileCheck %s
+
+#if !__has_feature(kcfi)
+#error Missing kcfi
+#endif
+
+// CHECK-LABEL: define dso_local i32 @f1(){{.*}} prefix i64 [[#%d,HASH:]]
+int f1(void) { return 0; }
+
+typedef int (*fn_t)(void);
+typedef int (*fn_unchecked_t)(void) __attribute__((kcfi_unchecked));
+
+typedef typeof(f1) *fn_typeof_t;
+typedef typeof(f1) *__attribute__((kcfi_unchecked)) fn_typeof_unchecked_t;
+
+// CHECK-LABEL: define{{.*}} i32 @checked()
+int checked(void) {
+ // CHECK: kcfi.fail
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]]
+ // CHECK: kcfi.cont
+ // CHECK: call i32 %
+ return ({ &f1; })();
+}
+// CHECK-LABEL: define{{.*}} i32 @checked_typedef_cast()
+int checked_typedef_cast(void) {
+ // CHECK: kcfi.fail
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]]
+ // CHECK: kcfi.cont
+ // CHECK: call i32 %
+ return ({ (fn_t) & f1; })();
+}
+// CHECK-LABEL: define{{.*}} i32 @checked_outside_typedef_cast()
+int checked_outside_typedef_cast(void) {
+ // CHECK: kcfi.fail
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]]
+ // CHECK: kcfi.cont
+ // CHECK: call i32 %
+ return ((fn_t)({ &f1; }))();
+}
+// CHECK-LABEL: define{{.*}} i32 @checked_typeof_typedef_cast()
+int checked_typeof_typedef_cast(void) {
+ // CHECK: kcfi.fail
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]]
+ // CHECK: kcfi.cont
+ // CHECK: call i32 %
+ return ({ (fn_typeof_t) & f1; })();
+}
+// CHECK-LABEL: define{{.*}} i32 @checked_var()
+int checked_var(void) {
+ fn_t p = f1;
+ // CHECK: kcfi.fail
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]]
+ // CHECK: kcfi.cont
+ // CHECK: call i32 %
+ return p();
+}
+// CHECK-LABEL: define{{.*}} i32 @checked_param(i32 ()*
+int checked_param(fn_t p) {
+ // CHECK: kcfi.fail
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]]
+ // CHECK: kcfi.cont
+ // CHECK: call i32 %
+ return p();
+}
+
+// CHECK-LABEL: define{{.*}} i32 @unchecked_typedef_cast()
+int unchecked_typedef_cast(void) {
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return ({ (fn_unchecked_t) & f1; })();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_outside_typedef_cast()
+int unchecked_outside_typedef_cast(void) {
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return ((fn_unchecked_t)({ &f1; }))();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_typeof_typedef_cast()
+int unchecked_typeof_typedef_cast(void) {
+ // CHECK-NOT: call void @__kcfi_check_fail
+ return ({ (fn_typeof_unchecked_t) & f1; })();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_compound_var()
+int unchecked_compound_var(void) {
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return ({
+ fn_unchecked_t p = f1;
+ p;
+ })();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_compound_local_typedef_cast()
+int unchecked_compound_local_typedef_cast(void) {
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return ({
+ typedef typeof(f1) *__attribute__((kcfi_unchecked)) fn_local_unchecked_t;
+ (fn_local_unchecked_t) & f1;
+ })();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_var()
+int unchecked_var(void) {
+ fn_unchecked_t p = f1;
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return p();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_var_attr()
+int unchecked_var_attr(void) {
+ fn_t __attribute__((kcfi_unchecked)) p = f1;
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return p();
+}
+// CHECK-LABEL: define{{.*}} i32 @unchecked_param(i32 ()*
+int unchecked_param(fn_t __attribute__((kcfi_unchecked)) p) {
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK: call i32 %
+ return p();
+}
Index: clang/test/CodeGen/kcfi.c
===================================================================
--- /dev/null
+++ clang/test/CodeGen/kcfi.c
@@ -0,0 +1,60 @@
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm -fsanitize=kcfi -o - %s | FileCheck --check-prefixes=CHECK,OFFSET0,FATAL %s
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm -fsanitize=kcfi -fsanitize-recover=kcfi -o - %s | FileCheck --check-prefixes=CHECK,OFFSET0,RECOVER %s
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm -fsanitize=kcfi -fsanitize-kcfi-offset=1 -fpatchable-function-entry=1 -fpatchable-function-entry-offset=1 -o - %s | FileCheck --check-prefixes=CHECK,OFFSET1 %s
+#if !__has_feature(kcfi)
+#error Missing kcfi?
+#endif
+
+// CHECK: module asm ".weak __kcfi_typeid_f4"
+// CHECK: module asm ".set __kcfi_typeid_f4, [[#%d,HASH:]]"
+
+typedef int (*fn_t)(void);
+
+// CHECK: define dso_local i32 @f1(){{.*}} prefix i64 [[#HASH]]
+int f1(void) { return 0; }
+
+// CHECK: define dso_local i32 @f2(){{.*}} prefix i64 [[#%d,HASH2:]]
+unsigned int f2(void) { return 2; }
+
+// CHECK-LABEL: define dso_local i32 @__call(i32 ()*{{.*}} %f)
+int __call(fn_t f) __attribute__((__no_sanitize__("kcfi"))) {
+ // CHECK-NOT: kcfi.fail
+ // CHECK-NOT: call void @__kcfi_check_fail
+ // CHECK-NOT: kcfi.cont
+ // CHECK: %call = call i32 %[[#]]()
+ return f();
+}
+
+// CHECK-LABEL: define dso_local i32 @call(i32 ()*{{.*}} %f)
+int call(fn_t f) {
+ // OFFSET1: %[[#I8CAST:]] = bitcast i32 ()* %{{.}} to i8*
+ // OFFSET1: %[[#OFFSET:]] = getelementptr inbounds i8, i8* %[[#I8CAST]], i64 -1
+ // OFFSET1: %[[#CAST:]] = bitcast i8* %[[#OFFSET]] to i64*
+ // OFFSET0: %[[#CAST:]] = bitcast i32 ()* %{{.}} to i64*
+ // CHECK: %[[#GEPI:]] = getelementptr inbounds i64, i64* %[[#CAST]], i64 -1
+ // CHECK: %[[#LOAD:]] = load i64, i64* %[[#GEPI]], align 8
+ // CHECK: %[[#ICMP:]] = icmp eq i64 %[[#LOAD]], [[#HASH]]
+ // CHECK: br i1 %[[#ICMP]], label %kcfi.cont, label %kcfi.fail
+ // CHECK-LABEL: kcfi.fail{{.*}}:
+ // CHECK: call void @__kcfi_check_fail(i64 [[#HASH]], i8*
+ // FATAL: unreachable
+ // RECOVER: br label %kcfi.cont
+ // CHECK-LABEL: kcfi.cont{{.*}}:
+ // CHECK: %call = call i32 %[[#]]()
+ return f();
+}
+
+// CHECK-DAG: define internal i32 @f3(){{.*}} prefix i64 [[#HASH]]
+static int f3(void) { return 1; }
+
+// CHECK-DAG: declare i32 @f4(){{.*}} prefix i64 [[#HASH]]
+extern int f4(void);
+
+// CHECK-DAG: declare void @__kcfi_check_fail(i64, i8*)
+
+int test() {
+ return call(f1) +
+ __call((fn_t)f2) +
+ call(f3) +
+ call(f4);
+}
Index: clang/lib/Sema/SemaDeclAttr.cpp
===================================================================
--- clang/lib/Sema/SemaDeclAttr.cpp
+++ clang/lib/Sema/SemaDeclAttr.cpp
@@ -4697,6 +4697,23 @@
D->addAttr(::new (Context) ModeAttr(Context, CI, Name));
}
+static void handleKCFIUncheckedAttr(Sema &S, Decl *D, const ParsedAttr &AL) {
+ QualType Type;
+
+ if (auto *TD = dyn_cast<TypedefNameDecl>(D))
+ Type = TD->getUnderlyingType();
+ else if (auto *VD = dyn_cast<VarDecl>(D))
+ if (auto *TSI = VD->getTypeSourceInfo())
+ Type = TSI->getType();
+
+ if (Type.isNull() || !Type->isFunctionPointerType()) {
+ S.Diag(D->getBeginLoc(), diag::err_attribute_function_pointers_only) << AL;
+ return;
+ }
+
+ D->addAttr(::new (S.Context) KCFIUncheckedAttr(S.Context, AL));
+}
+
static void handleNoDebugAttr(Sema &S, Decl *D, const ParsedAttr &AL) {
D->addAttr(::new (S.Context) NoDebugAttr(S.Context, AL));
}
@@ -8387,6 +8404,9 @@
case ParsedAttr::AT_Mode:
handleModeAttr(S, D, AL);
break;
+ case ParsedAttr::AT_KCFIUnchecked:
+ handleKCFIUncheckedAttr(S, D, AL);
+ break;
case ParsedAttr::AT_NonNull:
if (auto *PVD = dyn_cast<ParmVarDecl>(D))
handleNonNullAttrParameter(S, PVD, AL);
Index: clang/lib/Driver/ToolChain.cpp
===================================================================
--- clang/lib/Driver/ToolChain.cpp
+++ clang/lib/Driver/ToolChain.cpp
@@ -1034,6 +1034,9 @@
getTriple().getArch() == llvm::Triple::arm || getTriple().isWasm() ||
getTriple().isAArch64())
Res |= SanitizerKind::CFIICall;
+ if (getTriple().getArch() == llvm::Triple::x86_64 ||
+ getTriple().isAArch64(64))
+ Res |= SanitizerKind::KCFI;
if (getTriple().getArch() == llvm::Triple::x86_64 ||
getTriple().isAArch64(64) || getTriple().isRISCV())
Res |= SanitizerKind::ShadowCallStack;
Index: clang/lib/Driver/SanitizerArgs.cpp
===================================================================
--- clang/lib/Driver/SanitizerArgs.cpp
+++ clang/lib/Driver/SanitizerArgs.cpp
@@ -37,7 +37,8 @@
static const SanitizerMask NotAllowedWithMinimalRuntime =
SanitizerKind::Function | SanitizerKind::Vptr;
static const SanitizerMask RequiresPIE =
- SanitizerKind::DataFlow | SanitizerKind::HWAddress | SanitizerKind::Scudo;
+ SanitizerKind::DataFlow | SanitizerKind::HWAddress | SanitizerKind::Scudo |
+ SanitizerKind::KCFI;
static const SanitizerMask NeedsUnwindTables =
SanitizerKind::Address | SanitizerKind::HWAddress | SanitizerKind::Thread |
SanitizerKind::Memory | SanitizerKind::DataFlow;
@@ -145,17 +146,17 @@
struct Ignorelist {
const char *File;
SanitizerMask Mask;
- } Ignorelists[] = {{"asan_ignorelist.txt", SanitizerKind::Address},
- {"hwasan_ignorelist.txt", SanitizerKind::HWAddress},
- {"memtag_ignorelist.txt", SanitizerKind::MemTag},
- {"msan_ignorelist.txt", SanitizerKind::Memory},
- {"tsan_ignorelist.txt", SanitizerKind::Thread},
- {"dfsan_abilist.txt", SanitizerKind::DataFlow},
- {"cfi_ignorelist.txt", SanitizerKind::CFI},
- {"ubsan_ignorelist.txt",
- SanitizerKind::Undefined | SanitizerKind::Integer |
- SanitizerKind::Nullability |
- SanitizerKind::FloatDivideByZero}};
+ } Ignorelists[] = {
+ {"asan_ignorelist.txt", SanitizerKind::Address},
+ {"hwasan_ignorelist.txt", SanitizerKind::HWAddress},
+ {"memtag_ignorelist.txt", SanitizerKind::MemTag},
+ {"msan_ignorelist.txt", SanitizerKind::Memory},
+ {"tsan_ignorelist.txt", SanitizerKind::Thread},
+ {"dfsan_abilist.txt", SanitizerKind::DataFlow},
+ {"cfi_ignorelist.txt", SanitizerKind::CFI | SanitizerKind::KCFI},
+ {"ubsan_ignorelist.txt",
+ SanitizerKind::Undefined | SanitizerKind::Integer |
+ SanitizerKind::Nullability | SanitizerKind::FloatDivideByZero}};
for (auto BL : Ignorelists) {
if (!(Kinds & BL.Mask))
@@ -165,7 +166,7 @@
llvm::sys::path::append(Path, "share", BL.File);
if (D.getVFS().exists(Path))
IgnorelistFiles.push_back(std::string(Path.str()));
- else if (BL.Mask == SanitizerKind::CFI && DiagnoseErrors)
+ else if (BL.Mask & SanitizerKind::CFI && DiagnoseErrors)
// If cfi_ignorelist.txt cannot be found in the resource dir, driver
// should fail.
D.Diag(clang::diag::err_drv_no_such_file) << Path;
@@ -676,11 +677,22 @@
<< "-fsanitize-cfi-cross-dso"
<< "-fsanitize-cfi-icall-generalize-pointers";
+ if (AllAddedKinds & SanitizerKind::KCFI && DiagnoseErrors)
+ D.Diag(diag::err_drv_argument_not_allowed_with) << "-fsanitize=cfi*"
+ << "-fsanitize=kcfi";
+
CfiCanonicalJumpTables =
Args.hasFlag(options::OPT_fsanitize_cfi_canonical_jump_tables,
options::OPT_fno_sanitize_cfi_canonical_jump_tables, true);
}
+ if (AllAddedKinds & SanitizerKind::KCFI)
+ if (Arg *A = Args.getLastArg(options::OPT_fsanitize_kcfi_offset)) {
+ StringRef S = A->getValue();
+ if (S.getAsInteger(0, KCFIOffset) && DiagnoseErrors)
+ D.Diag(clang::diag::err_drv_invalid_value) << A->getAsString(Args) << S;
+ }
+
Stats = Args.hasFlag(options::OPT_fsanitize_stats,
options::OPT_fno_sanitize_stats, false);
@@ -1134,6 +1146,10 @@
if (CfiCanonicalJumpTables)
CmdArgs.push_back("-fsanitize-cfi-canonical-jump-tables");
+ if (KCFIOffset)
+ CmdArgs.push_back(
+ Args.MakeArgString("-fsanitize-kcfi-offset=" + Twine(KCFIOffset)));
+
if (Stats)
CmdArgs.push_back("-fsanitize-stats");
Index: clang/lib/CodeGen/CodeGenModule.h
===================================================================
--- clang/lib/CodeGen/CodeGenModule.h
+++ clang/lib/CodeGen/CodeGenModule.h
@@ -1374,6 +1374,9 @@
/// Generate a cross-DSO type identifier for MD.
llvm::ConstantInt *CreateCrossDsoCfiTypeId(llvm::Metadata *MD);
+ /// Generate a KCFI type identifier for T.
+ llvm::ConstantInt *CreateKCFITypeId(QualType T);
+
/// Create a metadata identifier for the given type. This may either be an
/// MDString (for external identifiers) or a distinct unnamed MDNode (for
/// internal identifiers).
@@ -1392,6 +1395,12 @@
void CreateFunctionTypeMetadataForIcall(const FunctionDecl *FD,
llvm::Function *F);
+ /// Set type hash as prefix data to the given function
+ void SetKCFITypePrefix(const FunctionDecl *FD, llvm::Function *F);
+
+ /// Emit KCFI type identifier constants
+ void EmitKCFIConstants();
+
/// Whether this function's return type has no side effects, and thus may
/// be trivially discarded if it is unused.
bool MayDropFunctionReturn(const ASTContext &Context, QualType ReturnType);
@@ -1613,7 +1622,8 @@
llvm::AttrBuilder &FuncAttrs);
llvm::Metadata *CreateMetadataIdentifierImpl(QualType T, MetadataTypeMap &Map,
- StringRef Suffix);
+ StringRef Suffix,
+ bool OnlyExternal = true);
};
} // end namespace CodeGen
Index: clang/lib/CodeGen/CodeGenModule.cpp
===================================================================
--- clang/lib/CodeGen/CodeGenModule.cpp
+++ clang/lib/CodeGen/CodeGenModule.cpp
@@ -45,6 +45,7 @@
#include "clang/Basic/Version.h"
#include "clang/CodeGen/ConstantInitBuilder.h"
#include "clang/Frontend/FrontendDiagnostic.h"
+#include "llvm/ADT/StringExtras.h"
#include "llvm/ADT/StringSwitch.h"
#include "llvm/ADT/Triple.h"
#include "llvm/Analysis/TargetLibraryInfo.h"
@@ -63,6 +64,7 @@
#include "llvm/Support/MD5.h"
#include "llvm/Support/TimeProfiler.h"
#include "llvm/Support/X86TargetParser.h"
+#include "llvm/Support/xxhash.h"
using namespace clang;
using namespace CodeGen;
@@ -546,6 +548,8 @@
CodeGenFunction(*this).EmitCfiCheckFail();
CodeGenFunction(*this).EmitCfiCheckStub();
}
+ if (LangOpts.Sanitize.has(SanitizerKind::KCFI))
+ EmitKCFIConstants();
emitAtAvailableLinkGuard();
if (Context.getTargetInfo().getTriple().isWasm() &&
!Context.getTargetInfo().getTriple().isOSEmscripten()) {
@@ -1582,6 +1586,17 @@
return llvm::ConstantInt::get(Int64Ty, llvm::MD5Hash(MDS->getString()));
}
+llvm::ConstantInt *CodeGenModule::CreateKCFITypeId(QualType T) {
+ llvm::MDString *MDS = dyn_cast<llvm::MDString>(CreateMetadataIdentifierImpl(
+ T, MetadataIdMap, "", /*OnlyExternal=*/false));
+ if (!MDS)
+ return nullptr;
+
+ return llvm::ConstantInt::get(
+ Int64Ty,
+ (llvm::xxHash64(MDS->getString()) & (-1ULL >> 16)) | (0x10CFULL << 48));
+}
+
void CodeGenModule::SetLLVMFunctionAttributes(GlobalDecl GD,
const CGFunctionInfo &Info,
llvm::Function *F, bool IsThunk) {
@@ -2168,6 +2183,49 @@
F->addTypeMetadata(0, llvm::ConstantAsMetadata::get(CrossDsoTypeId));
}
+void CodeGenModule::SetKCFITypePrefix(const FunctionDecl *FD,
+ llvm::Function *F) {
+
+ if (isa<CXXMethodDecl>(FD) && !cast<CXXMethodDecl>(FD)->isStatic())
+ return;
+
+ F->setPrefixData(CreateKCFITypeId(FD->getType()));
+}
+
+static bool allowKCFIIdentifier(const std::string &Name) {
+ // KCFI type identifier constants are only necessary for external assembly
+ // functions, which means it's safe to skip unusual names. Subset of
+ // MCAsmInfo::isAcceptableChar() and MCAsmInfoXCOFF::isAcceptableChar().
+ for (const char &C : Name) {
+ if (llvm::isAlnum(C) || C == '_' || C == '.')
+ continue;
+ return false;
+ }
+ return true;
+}
+
+void CodeGenModule::EmitKCFIConstants() {
+ llvm::Module &M = getModule();
+ // Generate a weak constant with the expected KCFI type identifier for all
+ // address-taken function declarations.
+ for (auto &F : M.functions()) {
+ if (!F.isDeclaration() || !F.hasAddressTaken() || !F.hasPrefixData())
+ continue;
+
+ auto *Id = dyn_cast<llvm::ConstantInt>(F.getPrefixData());
+ if (!Id)
+ continue;
+
+ std::string Name = "__kcfi_typeid_" + F.getName().str();
+ if (!allowKCFIIdentifier(Name))
+ continue;
+
+ std::string Asm = ".weak " + Name + "\n" + ".set " + Name + ", " +
+ std::to_string(Id->getSExtValue()) + "\n";
+ M.appendModuleInlineAsm(Asm);
+ }
+}
+
void CodeGenModule::SetFunctionAttributes(GlobalDecl GD, llvm::Function *F,
bool IsIncompleteFunction,
bool IsThunk) {
@@ -2250,6 +2308,9 @@
!CodeGenOpts.SanitizeCfiCanonicalJumpTables)
CreateFunctionTypeMetadataForIcall(FD, F);
+ if (LangOpts.Sanitize.has(SanitizerKind::KCFI))
+ SetKCFITypePrefix(FD, F);
+
if (getLangOpts().OpenMP && FD->hasAttr<OMPDeclareSimdDeclAttr>())
getOpenMPRuntime().emitDeclareSimdFunction(FD, F);
@@ -6430,7 +6491,8 @@
llvm::Metadata *
CodeGenModule::CreateMetadataIdentifierImpl(QualType T, MetadataTypeMap &Map,
- StringRef Suffix) {
+ StringRef Suffix,
+ bool OnlyExternal /*=true*/) {
if (auto *FnType = T->getAs<FunctionProtoType>())
T = getContext().getFunctionType(
FnType->getReturnType(), FnType->getParamTypes(),
@@ -6440,7 +6502,7 @@
if (InternalId)
return InternalId;
- if (isExternallyVisible(T->getLinkage())) {
+ if (isExternallyVisible(T->getLinkage()) || !OnlyExternal) {
std::string OutName;
llvm::raw_string_ostream Out(OutName);
getCXXABI().getMangleContext().mangleTypeName(T, Out);
Index: clang/lib/CodeGen/CodeGenFunction.h
===================================================================
--- clang/lib/CodeGen/CodeGenFunction.h
+++ clang/lib/CodeGen/CodeGenFunction.h
@@ -4591,6 +4591,8 @@
/// passing to a runtime sanitizer handler.
llvm::Constant *EmitCheckSourceLocation(SourceLocation Loc);
+ void EmitKCFICheck(llvm::Value *Ptr, llvm::ConstantInt *Hash);
+
/// Create a basic block that will either trap or call a handler function in
/// the UBSan runtime with the provided arguments, and create a conditional
/// branch to it.
Index: clang/lib/CodeGen/CGExpr.cpp
===================================================================
--- clang/lib/CodeGen/CGExpr.cpp
+++ clang/lib/CodeGen/CGExpr.cpp
@@ -3151,6 +3151,48 @@
#undef SANITIZER_CHECK
};
+void CodeGenFunction::EmitKCFICheck(llvm::Value *Ptr, llvm::ConstantInt *Hash) {
+ uint64_t Offset = CGM.getCodeGenOpts().SanitizeKCFIOffset;
+ if (Offset)
+ Ptr = Builder
+ .CreateConstInBoundsGEP(
+ Address(Builder.CreateBitCast(Ptr, Int8PtrTy),
+ CGM.getPointerAlign()),
+ -Offset)
+ .getPointer();
+
+ Address HashPtr = Builder.CreateConstInBoundsGEP(
+ Address(Builder.CreateBitCast(Ptr, Int64Ty->getPointerTo(0)),
+ CGM.getPointerAlign()),
+ -1);
+ llvm::Value *Test = Builder.CreateICmpEQ(Builder.CreateLoad(HashPtr), Hash);
+ llvm::BasicBlock *ContBB = createBasicBlock("kcfi.cont");
+ llvm::BasicBlock *FailBB = createBasicBlock("kcfi.fail");
+ llvm::Instruction *Branch = Builder.CreateCondBr(Test, ContBB, FailBB);
+
+ llvm::MDBuilder MDHelper(getLLVMContext());
+ Branch->setMetadata(llvm::LLVMContext::MD_prof,
+ MDHelper.createBranchWeights((1U << 20) - 1, 1));
+
+ EmitBlock(FailBB);
+
+ llvm::FunctionCallee Handler = CGM.getModule().getOrInsertFunction(
+ "__kcfi_check_fail",
+ llvm::FunctionType::get(VoidTy, {Int64Ty, Int8PtrTy}, false));
+ llvm::CallInst *Call = Builder.CreateCall(
+ Handler, {Hash, Builder.CreateBitCast(Ptr, Int8PtrTy)});
+
+ if (CGM.getCodeGenOpts().SanitizeRecover.has(SanitizerKind::KCFI))
+ Builder.CreateBr(ContBB);
+ else {
+ Call->setDoesNotReturn();
+ Call->setDoesNotThrow();
+ Builder.CreateUnreachable();
+ }
+
+ EmitBlock(ContBB);
+}
+
static void emitCheckHandlerCall(CodeGenFunction &CGF,
llvm::FunctionType *FnType,
ArrayRef<llvm::Value *> FnArgs,
@@ -5164,6 +5206,7 @@
const Decl *TargetDecl =
OrigCallee.getAbstractInfo().getCalleeDecl().getDecl();
+ auto OrigType = CalleeType;
CalleeType = getContext().getCanonicalType(CalleeType);
auto PointeeType = cast<PointerType>(CalleeType)->getPointeeType();
@@ -5220,11 +5263,11 @@
}
const auto *FnType = cast<FunctionType>(PointeeType);
+ bool IsIndirectCall = !TargetDecl || !isa<FunctionDecl>(TargetDecl);
// If we are checking indirect calls and this call is indirect, check that the
// function pointer is a member of the bit set for the function type.
- if (SanOpts.has(SanitizerKind::CFIICall) &&
- (!TargetDecl || !isa<FunctionDecl>(TargetDecl))) {
+ if (SanOpts.has(SanitizerKind::CFIICall) && IsIndirectCall) {
SanitizerScope SanScope(this);
EmitSanitizerStatReport(llvm::SanStat_CFI_ICall);
@@ -5257,6 +5300,14 @@
}
}
+ if (SanOpts.has(SanitizerKind::KCFI) && IsIndirectCall) {
+ auto *TargetType = dyn_cast<TypedefType>(OrigType);
+ if ((!TargetDecl || !TargetDecl->hasAttr<KCFIUncheckedAttr>()) &&
+ (!TargetType || !TargetType->getDecl()->hasAttr<KCFIUncheckedAttr>()))
+ EmitKCFICheck(Callee.getFunctionPointer(),
+ CGM.CreateKCFITypeId(QualType(FnType, 0)));
+ }
+
CallArgList Args;
if (Chain)
Args.add(RValue::get(Builder.CreateBitCast(Chain, CGM.VoidPtrTy)),
Index: clang/include/clang/Driver/SanitizerArgs.h
===================================================================
--- clang/include/clang/Driver/SanitizerArgs.h
+++ clang/include/clang/Driver/SanitizerArgs.h
@@ -48,6 +48,7 @@
bool AsanOutlineInstrumentation = false;
llvm::AsanDtorKind AsanDtorKind = llvm::AsanDtorKind::Invalid;
std::string HwasanAbi;
+ unsigned int KCFIOffset = 0;
bool LinkRuntimes = true;
bool LinkCXXRuntimes = false;
bool NeedPIE = false;
Index: clang/include/clang/Driver/Options.td
===================================================================
--- clang/include/clang/Driver/Options.td
+++ clang/include/clang/Driver/Options.td
@@ -1747,6 +1747,11 @@
PosFlag<SetTrue, [], "Make">, NegFlag<SetFalse, [CoreOption, NoXarchOption], "Do not make">,
BothFlags<[], " the jump table addresses canonical in the symbol table">>,
Group<f_clang_Group>;
+def fsanitize_kcfi_offset
+ : Joined<["-"], "fsanitize-kcfi-offset=">, MetaVarName<"<value>">,
+ Group<f_clang_Group>,
+ HelpText<"Assume there are <value> bytes between the KCFI type identifer and start of the function.">,
+ MarshallingInfoInt<CodeGenOpts<"SanitizeKCFIOffset">>;
defm sanitize_stats : BoolOption<"f", "sanitize-stats",
CodeGenOpts<"SanitizeStats">, DefaultFalse,
PosFlag<SetTrue, [], "Enable">, NegFlag<SetFalse, [CoreOption, NoXarchOption], "Disable">,
Index: clang/include/clang/Basic/Sanitizers.def
===================================================================
--- clang/include/clang/Basic/Sanitizers.def
+++ clang/include/clang/Basic/Sanitizers.def
@@ -124,6 +124,9 @@
CFIDerivedCast | CFIICall | CFIMFCall | CFIUnrelatedCast |
CFINVCall | CFIVCall)
+// Kernel Control Flow Integrity
+SANITIZER("kcfi", KCFI)
+
// Safe Stack
SANITIZER("safe-stack", SafeStack)
Index: clang/include/clang/Basic/Features.def
===================================================================
--- clang/include/clang/Basic/Features.def
+++ clang/include/clang/Basic/Features.def
@@ -223,6 +223,7 @@
FEATURE(is_trivially_copyable, LangOpts.CPlusPlus)
FEATURE(is_union, LangOpts.CPlusPlus)
FEATURE(modules, LangOpts.Modules)
+FEATURE(kcfi, LangOpts.Sanitize.has(SanitizerKind::KCFI))
FEATURE(safe_stack, LangOpts.Sanitize.has(SanitizerKind::SafeStack))
FEATURE(shadow_call_stack,
LangOpts.Sanitize.has(SanitizerKind::ShadowCallStack))
Index: clang/include/clang/Basic/DiagnosticSemaKinds.td
===================================================================
--- clang/include/clang/Basic/DiagnosticSemaKinds.td
+++ clang/include/clang/Basic/DiagnosticSemaKinds.td
@@ -3056,6 +3056,9 @@
def warn_attribute_pointer_or_reference_only : Warning<
"%0 attribute only applies to a pointer or reference (%1 is invalid)">,
InGroup<IgnoredAttributes>;
+def err_attribute_function_pointers_only : Error<
+ "%0 attribute argument only applies to a function pointer or a function "
+ "pointer type">;
def err_attribute_no_member_pointers : Error<
"%0 attribute cannot be used with pointers to members">;
def err_attribute_invalid_implicit_this_argument : Error<
Index: clang/include/clang/Basic/CodeGenOptions.def
===================================================================
--- clang/include/clang/Basic/CodeGenOptions.def
+++ clang/include/clang/Basic/CodeGenOptions.def
@@ -248,6 +248,8 @@
///< CFI icall function signatures
CODEGENOPT(SanitizeCfiCanonicalJumpTables, 1, 0) ///< Make jump table symbols canonical
///< instead of creating a local jump table.
+CODEGENOPT(SanitizeKCFIOffset, 32, 0) ///< Define the KCFI type identifier offset from
+ ///< the start of the function.
CODEGENOPT(SanitizeCoverageType, 2, 0) ///< Type of sanitizer coverage
///< instrumentation.
CODEGENOPT(SanitizeCoverageIndirectCalls, 1, 0) ///< Enable sanitizer coverage
Index: clang/include/clang/Basic/AttrDocs.td
===================================================================
--- clang/include/clang/Basic/AttrDocs.td
+++ clang/include/clang/Basic/AttrDocs.td
@@ -5399,6 +5399,32 @@
}];
}
+def KCFIUncheckedDocs : Documentation {
+ let Category = DocCatType;
+ let Content = [{
+The ``kcfi_unchecked`` attribute causes Clang to not emit KCFI checks for indirect
+calls made through annotated function pointer variables or types.
+
+.. code-block:: c
+
+ int f1(void) { return 0; }
+
+ int (*p1)(void) = f1;
+ p1(); // checked
+
+ int (*p2)(void) __attribute__((kcfi_unchecked)) = f1;
+ p2(); // unchecked
+
+ typedef typeof(f1) * __attribute__((kcfi_unchecked) unchecked_t;
+ ((unchecked_t)p2)(); // unchecked
+
+ unchecked_t p3 = f1;
+ p3(); // unchecked
+
+``kcfi_unchecked`` is only supported for function pointers and function pointer types.
+}];
+}
+
def ReinitializesDocs : Documentation {
let Category = DocCatFunction;
let Content = [{
Index: clang/include/clang/Basic/Attr.td
===================================================================
--- clang/include/clang/Basic/Attr.td
+++ clang/include/clang/Basic/Attr.td
@@ -692,6 +692,12 @@
let Documentation = [AlignValueDocs];
}
+def KCFIUnchecked : Attr {
+ let Spellings = [Clang<"kcfi_unchecked">];
+ let Subjects = SubjectList<[Var, TypedefName]>;
+ let Documentation = [KCFIUncheckedDocs];
+}
+
def AlignMac68k : InheritableAttr {
// This attribute has no spellings as it is only ever created implicitly.
let Spellings = [];
Index: clang/docs/UsersManual.rst
===================================================================
--- clang/docs/UsersManual.rst
+++ clang/docs/UsersManual.rst
@@ -1666,6 +1666,8 @@
flow analysis.
- ``-fsanitize=cfi``: :doc:`control flow integrity <ControlFlowIntegrity>`
checks. Requires ``-flto``.
+ - ``-fsanitize=kcfi``: kernel indirect call forward-edge control flow
+ integrity.
- ``-fsanitize=safe-stack``: :doc:`safe stack <SafeStack>`
protection against stack-based memory corruption errors.
@@ -3939,6 +3941,8 @@
Enable control flow integrity (CFI) checks for cross-DSO calls.
-fsanitize-cfi-icall-generalize-pointers
Generalize pointers in CFI indirect call type signature checks
+ -fsanitize-kcfi-offset=<value>
+ Offset between the KCFI type identifier and the start of a function.
-fsanitize-coverage=<value>
Specify the type of coverage instrumentation for Sanitizers
-fsanitize-hwaddress-abi=<value>
Index: clang/docs/ClangCommandLineReference.rst
===================================================================
--- clang/docs/ClangCommandLineReference.rst
+++ clang/docs/ClangCommandLineReference.rst
@@ -959,6 +959,10 @@
Generalize pointers in CFI indirect call type signature checks
+.. option:: -fsanitize-kcfi-offset=<value>
+
+Set the number of bytes between the KCFI type identifier and the start of the function.
+
.. option:: -fsanitize-coverage-allowlist=<arg>, -fsanitize-coverage-whitelist=<arg>
Restrict sanitizer coverage instrumentation exclusively to modules and functions that match the provided special case list, except the blocked ones
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
