| Issue |
203764
|
| Summary |
[clang][Diagnostics] Preprocessor builds a per-token CheckPoints index on every Lex() even when no diagnostic ever needs it
|
| Labels |
clang
|
| Assignees |
|
| Reporter |
AnonMiraj
|
While benchmarking Boost.MPL compile times across Clang versions, I noticed a consistent ~0.4% instruction count regression jumping from Clang 18 to 19. I tracked this down to commit `718aac9f` ("Highlight code snippets", #66514).
| Commit | Basket Ir | Delta |
| --- | --- | --- |
| `863b2c84` (Parent) | 30,222,222,121 | — |
| `718aac9f` (**Commit**) | **30,345,311,902** | **+0.407%** |
The issue is that `Preprocessor::Lex()` now builds a checkpoint index on the hot path for every single token, regardless of whether a diagnostic actually needs it. Every token pays for an increment and a branch, and every 1024 tokens triggers a `DenseMap` insert and `SmallVector` push. For token-heavy code like template metaprogramming, this adds up quickly on clean runs that emit zero warnings or errors.
To confirm this loop was the sole culprit, I re-measured the same commit with just the checkpoint tracking block disabled. The data shows it accounts for the entire regression:
| Configuration | Basket Ir | Delta |
| --- | --- | --- |
| `718aac9f` | 30,344,081,437 | — |
| Same, Checkpoint Disabled | 30,214,317,878 | — |
| **Difference** | **−129,763,559** | **−0.43%** |
Instead of building everything upfront, we should try to do it lazily as much as possible.
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
