Issue 208751
Summary [X86][CET] isEndbrImm64 checks byte-reversed ENDBR immediates under -fcf-protection=branch
Labels
Assignees
Reporter BIGSMATER
    ### Description

This was first submitted through the LLVM Security Response process. The group
triaged it as a regular bug and closed the private ticket, so I am filing it
publicly here.

The X86 CET/IBT immediate scrubber in
`X86DAGToDAGISel::PreprocessISelDAG()` appears to check ENDBR immediates in
byte-display order instead of the integer value whose little-endian encoding
produces the ENDBR byte sequence in `.text`.

For ENDBR64, the byte sequence is:

```text
f3 0f 1e fa
```

On little-endian x86, the 32-bit integer immediate that emits those bytes is:

```text
0xfa1e0ff3
```

However, the current scrubber checks `0xf30f1efa`. As a result, `0xf30f1efa`
is rewritten, but `0xfa1e0ff3` is emitted as a single immediate and leaves an
unintended `f3 0f 1e fa` byte sequence inside the instruction stream.

I originally reproduced this with `llvmorg-22.1.4`. I also checked current
`main` source on 2026-07-10 at commit
`3c138b5c5958dce2daefa923a1cae612e47a4273`; before applying a local fix, the
relevant source still used the same byte-display constants.

Relevant source:

- `llvm/lib/Target/X86/X86ISelDAGToDAG.cpp`
- `llvm/lib/Target/X86/X86AsmPrinter.cpp`

The same backend has a reference that appears to use the correct byte order:
`X86AsmPrinter::MaskKCFIType()` lists `0xFA1E0FF3` for ENDBR64 and
`0xFB1E0FF3` for ENDBR32.

### Reduced test case

```c
// trigger.c

unsigned control_rewritten(void) {
 return 0xF30F1EFAU;
}

unsigned endbr64_bytes(void) {
  return 0xFA1E0FF3U;
}

void store_endbr64_bytes(unsigned *p) {
  *p = 0xFA1E0FF3U;
}

unsigned endbr32_bytes(void) {
  return 0xFB1E0FF3U;
}
```

### Steps to reproduce

Use `clang` and `llvm-objdump` from an affected LLVM build.

```sh
clang -O2 -fcf-protection=branch -c trigger.c -o trigger.o
llvm-objdump -d --no-show-raw-insn=false trigger.o
```

Optional 32-bit check, if multilib support is available:

```sh
clang -m32 -O2 -fcf-protection=branch -c trigger.c -o trigger32.o
llvm-objdump -d --no-show-raw-insn=false trigger32.o
```

### Actual behavior

With `llvmorg-22.1.4`, `0xF30F1EFA` is rewritten through a complement
sequence:

```text
0000000000000000 <control_rewritten>:
 0: f3 0f 1e fa                  endbr64
       4: b8 05 e1 f0 0c movl   $0xcf0e105, %eax
       9: f7 d0                        notl %eax
       b: c3                           retq
```

But `0xFA1E0FF3` is emitted directly:

```text
0000000000000010 <endbr64_bytes>:
      10: f3 0f 1e fa                  endbr64
      14: b8 f3 0f 1e fa movl   $0xfa1e0ff3, %eax
      19: c3 retq
```

The bytes at offset `0x15` are:

```text
f3 0f 1e fa
```

Those bytes are not the intentional function-entry ENDBR64 at offset `0x10`;
they occur inside the `movl` immediate.

The same issue also appears for stores:

```text
0000000000000030 <store_endbr64_bytes>:
      30: f3 0f 1e fa                  endbr64
 34: c7 07 f3 0f 1e fa            movl   $0xfa1e0ff3, (%rdi)
      3a: c3 retq
```

The ENDBR32 byte pattern also passes through as a direct immediate in the
affected code path:

```text
0000000000000040 <endbr32_bytes>:
      40: f3 0f 1e fa endbr64
      44: b8 f3 0f 1e fb               movl $0xfb1e0ff3, %eax
      49: c3                           retq
```

### Expected behavior

When `-fcf-protection=branch` is enabled, the X86 backend should avoid emitting
unintended ENDBR32/ENDBR64 byte sequences in executable code outside
intentional landing pads.

In particular, `0xFA1E0FF3` should not be emitted as a single little-endian
32-bit immediate in `.text`, because its bytes are `f3 0f 1e fa`.

The compiler should materialize the value using a split/complement sequence, or
another encoding that does not leave the ENDBR byte sequence in the instruction
stream.

### Relevant source

The affected code appears to be in:

```text
llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
```

`isEndbrImm64()` currently checks for low 24 bits `0x0F1EFA` and then searches
higher bytes for `0xF3`.

`PreprocessISelDAG()` currently uses:

```c++
int32_t EndbrImm = Subtarget->is64Bit() ? 0xF30F1EFA : 0xF30F1EFB;
```

Those constants are the ENDBR opcode bytes read left-to-right, not the integer
values whose little-endian encodings produce those bytes.

### Environment

Original reproducer:

```text
clang version 22.1.4
source anchor: llvmorg-22.1.4, 08436c2608214f7ae6c7bb6fa175f432c2602778
build config: optimized build with assertions
target: x86_64-unknown-linux-gnu
host: Linux 6.18.33.2-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC Thu Jun 18 21:54:43 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
```

Current main check and candidate fix:

```text
llvm-project main commit checked: 3c138b5c5958dce2daefa923a1cae612e47a4273
candidate fix branch: https://github.com/BIGSMATER/llvm-project/tree/fix-x86-endbr-immediate-scrub
candidate fix commit: aadf99290ea52af2ab8100ecfc3221451bbecda4
candidate build: Release, X86 target only, assertions enabled
candidate llc version: LLVM 23.0.0git, optimized build with assertions
```

Candidate fix verification:

```text
ninja -C /tmp/llvm-project-full-pr-build llc FileCheck not count llvm-config llvm-readobj
/tmp/llvm-project-full-pr-build/bin/llvm-lit -sv /tmp/llvm-project-full-pr/llvm/test/CodeGen/X86/cet_endbr_imm_enhance.ll
git diff --check
```

The single updated regression test passed:

```text
Total Discovered Tests: 1
  Passed: 1 (100.00%)
```

### Candidate fix

I have a candidate patch that updates the existing X86 CET immediate regression
test and changes the scrubber to scan the bytes as they are emitted in
little-endian immediate operands:

```text
https://github.com/BIGSMATER/llvm-project/tree/fix-x86-endbr-immediate-scrub
```
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to