Meinersbur created this revision.
Meinersbur added reviewers: hfinkel, kbarton, SjoerdMeijer, aaron.ballman,
ABataev, fhahn, hsaito, hans, greened, dmgreen, reames, Ayal, asavonic, rtrieu,
dorit, rsmith, tyler.nowicki.
Herald added a reviewer: bollu.
Herald added a reviewer: jdoerfert.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
This is a series of patches that adds a new pragma for loop transformations. I
hope for feedback before the LLVM DevMtg where this will be the topic of my
talk <http://llvm.org/devmtg/2019-10/talk-abstracts.html#tech20>. The talk will
give an overview about how to add such an extension that touches all of clang's
layers and would hate to give wrong advice.
The syntax is:
#pragma clang transform distribute
#pragma clang transform unroll/unrollandjam [full/partial(n)]
#pragma clang transform vectorize [width(n)]
#pragma clang transform interleave [factor(n)]
The selection is currently limited to the passes LLVM currently supports. I am
working on more transformations that currently are only picked-up by Polly. The
biggest difference to #pragma clang loop it allows to specify in which order
the transformations are applied, which is ignored by clang's current LoopHint
attribute. It is also designed a bit more carefully, e.g. vectorize and
interleave are unambiguously different transformations and no question whether
setting an optimization option also enables the transformations.
In the longer term, we plan to add more features such as:
- More transformations (tiling, fusion, interchange, array packing, reversal,
wavefronting, peeling, splitting, space-filling curves, unswitching,
collapsing, strip/strip-mining, blocking, )
- More options
- Assigning identifiers to code such that they can be referenced by
transformations. (e.g. tile a loop nest, vectorize the second-to-innermost loop
and parallelize the outermost).
- Non-loop transformations (code motion, versioning, ...)
- OpenMP compatibility
Regarding the latter item, we are adding loop transformation to the OpenMP
specification. The next technical report presented at SC'19 will feature a
tiling transformation. As such, this patch is inspired by clang's OpenMP
implementation to make an integration later easier. It's not OpenMP though, in
that for instance the OpenMP construct will apply tiling regardless of semantic
equivalence while `#pragma clang transform` takes the classical compiler-hint
approach in that it (by default) still does a correctness check, only
influencing the profitability heuristic.
A previous prototype that was closer to how `#pragma clang loop` is implemented
using attributes instead of adding an additional kind of AST nodes. This showed
its limitations in that it did not allow all use-cases (such as #pragma without
a following statement) and its argument format can only store an array of
in-source identifiers and expressions. The prototype also used the `'#pragma
clang loop` syntax, but it proved difficult to disambiguate whether the
transformations are ordered or not.
The patch is split into multiple reviews:
- [this patch] The lexer part adds annotation begin- and end-tokens to the
token stream, as OpenMP does.
- The parser part parses the tokens between the annotation tokens and calls
`ActOn...` methods of Sema which are empty in this patch.
- The sema part also adds the AST nodes kinds: the Stmt representing the
#pragma, the clauses and `Transform`. The latter represents the semantic
analysis result similar to an ImplicitCast. Moreover, the AnalysisTransform
component constructs a loop nest tree to which transformations are applied to
such that Sema can warn about inconsistencies, e.g. there is no inner or
ambiguous loops for unrollandjam.
- The codegen part uses the same AnalysisTransform to determine which loop
metadata nodes to emit.
- Other parts not yet ready for a review: serialization, index, completion,
libclang, ASTMatcher, tooling
Thanks in advance for the review!
Repository:
rC Clang
https://reviews.llvm.org/D69088
Files:
include/clang/Basic/TokenKinds.def
include/clang/Parse/Parser.h
lib/Parse/ParsePragma.cpp
Index: lib/Parse/ParsePragma.cpp
===================================================================
--- lib/Parse/ParsePragma.cpp
+++ lib/Parse/ParsePragma.cpp
@@ -225,6 +225,12 @@
Token &FirstToken) override;
};
+struct PragmaTransformHandler : public PragmaHandler {
+ PragmaTransformHandler() : PragmaHandler("transform") {}
+ void HandlePragma(Preprocessor &PP, PragmaIntroducer Introducer,
+ Token &FirstToken) override;
+};
+
struct PragmaMSRuntimeChecksHandler : public EmptyPragmaHandler {
PragmaMSRuntimeChecksHandler() : EmptyPragmaHandler("runtime_checks") {}
};
@@ -376,6 +382,9 @@
std::make_unique<PragmaUnrollHintHandler>("nounroll_and_jam");
PP.AddPragmaHandler(NoUnrollAndJamHintHandler.get());
+ TransformHandler = std::make_unique<PragmaTransformHandler>();
+ PP.AddPragmaHandler("clang", TransformHandler.get());
+
FPHandler = std::make_unique<PragmaFPHandler>();
PP.AddPragmaHandler("clang", FPHandler.get());
@@ -482,6 +491,9 @@
PP.RemovePragmaHandler(NoUnrollAndJamHintHandler.get());
NoUnrollAndJamHintHandler.reset();
+ PP.RemovePragmaHandler("clang", TransformHandler.get());
+ TransformHandler.reset();
+
PP.RemovePragmaHandler("clang", FPHandler.get());
FPHandler.reset();
@@ -3008,6 +3020,71 @@
/*DisableMacroExpansion=*/false, /*IsReinject=*/false);
}
+/// Handle
+/// #pragma clang transform ...
+void PragmaTransformHandler::HandlePragma(Preprocessor &PP,
+ PragmaIntroducer Introducer,
+ Token &FirstTok) {
+ // "clang" token is not passed
+ // "transform" is FirstTok
+ // Everything up until tok::eod (or tok::eof) is wrapped between
+ // tok::annot_pragma_transform and tok::annot_pragma_transform_end, and
+ // pushed-back into the token stream. The tok::eod/eof is consumed as well:
+ //
+ // Token stream before:
+ // FirstTok:"transform" | <trans> [clauses..] eod ...
+ //
+ // Token stream after :
+ // "transform" <trans> [clauses..] eod | ...
+ // After pushing the annotation tokens:
+ //
+ // | annot_pragma_transform <trans> [clauses..] annot_pragma_transform_end ...
+ //
+ // The symbol | is before the next token returned by PP.Lex()
+ SmallVector<Token, 16> PragmaToks;
+
+ Token StartTok;
+ StartTok.startToken();
+ StartTok.setKind(tok::annot_pragma_transform);
+ StartTok.setLocation(FirstTok.getLocation());
+ PragmaToks.push_back(StartTok);
+
+ SourceLocation EodLoc = FirstTok.getLocation();
+ while (true) {
+ Token Tok;
+ PP.Lex(Tok);
+
+ // TODO: Handle nested pragmas as in r325369.
+ assert(!Tok.isAnnotation());
+ assert(Tok.isNot(tok::annot_pragma_transform));
+ assert(Tok.isNot(tok::annot_pragma_transform_end));
+ assert(Tok.isNot(tok::annot_pragma_openmp));
+ assert(Tok.isNot(tok::annot_pragma_openmp_end));
+ assert(Tok.isNot(tok::annot_pragma_loop_hint));
+
+ if (Tok.is(tok::eod) || Tok.is(tok::eof)) {
+ EodLoc = Tok.getLocation();
+ break;
+ }
+
+ PragmaToks.push_back(Tok);
+ }
+
+ Token EndTok;
+ EndTok.startToken();
+ EndTok.setKind(tok::annot_pragma_transform_end);
+ EndTok.setLocation(EodLoc);
+ PragmaToks.push_back(EndTok);
+
+ // Copy tokens for the preprocessor to own and free.
+ auto Toks = std::make_unique<Token[]>(PragmaToks.size());
+ std::copy(PragmaToks.begin(), PragmaToks.end(), Toks.get());
+
+ // Handle in parser
+ PP.EnterTokenStream(std::move(Toks), PragmaToks.size(),
+ /*DisableMacroExpansion=*/false, /*IsReinject=*/false);
+}
+
/// Handle the Microsoft \#pragma intrinsic extension.
///
/// The syntax is:
Index: include/clang/Parse/Parser.h
===================================================================
--- include/clang/Parse/Parser.h
+++ include/clang/Parse/Parser.h
@@ -195,6 +195,7 @@
std::unique_ptr<PragmaHandler> NoUnrollHintHandler;
std::unique_ptr<PragmaHandler> UnrollAndJamHintHandler;
std::unique_ptr<PragmaHandler> NoUnrollAndJamHintHandler;
+ std::unique_ptr<PragmaHandler> TransformHandler;
std::unique_ptr<PragmaHandler> FPHandler;
std::unique_ptr<PragmaHandler> STDCFENVHandler;
std::unique_ptr<PragmaHandler> STDCCXLIMITHandler;
Index: include/clang/Basic/TokenKinds.def
===================================================================
--- include/clang/Basic/TokenKinds.def
+++ include/clang/Basic/TokenKinds.def
@@ -828,6 +828,11 @@
PRAGMA_ANNOTATION(pragma_openmp)
PRAGMA_ANNOTATION(pragma_openmp_end)
+// Annotations for code transformation pragmas
+// #pragma clang transform ...
+ANNOTATION(pragma_transform)
+ANNOTATION(pragma_transform_end)
+
// Annotations for loop pragma directives #pragma clang loop ...
// The lexer produces these so that they only take effect when the parser
// handles #pragma loop ... directives.
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
