Hahnfeld marked 2 inline comments as done.
Hahnfeld added inline comments.
================
Comment at: clang/lib/Lex/Preprocessor.cpp:998
+std::vector<Token> Preprocessor::LexAll() {
+ std::vector<Token> toks;
----------------
aaron.ballman wrote:
> v.g.vassilev wrote:
> > Shouldn't we take the results as a `LexAll(std::vector<Token> &result)`?
> >
> > Perhaps we should rename this interface to `LexTokensUntilEOF`?
> I think both suggestions make sense, especially given this doesn't return the
> EOF token (so it doesn't really lex *all*). Though, given how many callers do
> not want to store the tokens, perhaps it should take a `std::optional` or a
> pointer to the vector so we can skip the storage work if it's unnecessary?
The new `LexTokensUntilEOF` now takes an optional `std::vector<Token> *Tokens =
nullptr`.
================
Comment at: clang/lib/Lex/Preprocessor.cpp:1000
+ std::vector<Token> toks;
+ while (1) {
+ Token tok;
----------------
aaron.ballman wrote:
> I'd prefer not to assume the token stream has an EOF token (perhaps the
> stream is one only being used to parse until the `eod` token instead), so if
> we can turn this into a non-infinite loop, that would make me more
> comfortable.
I'm not sure I understand entirely. Do you want something like
```
tok.isOneOf(tok::unknown, tok::eof, tok::eod)
```
instead of `tok.is(tok::eof)`? Can this happen at the level of the
`Preprocessor`?
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D158413/new/
https://reviews.llvm.org/D158413
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
