[PATCH] D152416: add -dump-tokens-bg option for clang

ChuanqiXu added a comment. In D152416#4404862 , @Anarion-zuo wrote: > I see that plugin is a more elegant way of doing this. Do you still want this > as a part of LLVM? No, I don't feel this is needed. Repository: rG LLVM Github Monorepo CHANGES S

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo added a comment. I see that plugin is a more elegant way of doing this. Do you still want this as a part of LLVM? Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D152416/new/ https://reviews.llvm.org/D152416 _

[PATCH] D152416: add -dump-tokens-bg option for clang

ChuanqiXu added a comment. I think such requirement should be done by something like a plugin or an analysis tool by using clang as a library instead of implementing it in clang compiler itself directly. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.or

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo added a comment. We intend to build a deep learning model with code as input, then noticed clang does not currently have a convenient way of dumping tokens in huge projects. The dumped tokens are supposed to be converted to vectors, then given to a deep learning model. Repository:

[PATCH] D152416: add -dump-tokens-bg option for clang

ChuanqiXu added a comment. What is the intended use case of this? Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D152416/new/ https://reviews.llvm.org/D152416 ___ cfe-commits mailing list cfe-commits@lists

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo added a comment. It's my first time here. I don't know if I got everything right. CodeOwners.rst says you guys are relevant somewhat. Thanks! @ChuanqiXu @jansvoboda11 Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D152416/new/ https://r

[PATCH] D152416: add -dump-tokens-bg option for clang

Anarion-zuo created this revision. Herald added a project: All. Anarion-zuo requested review of this revision. Herald added a project: clang. Herald added a subscriber: cfe-commits. Repository: rG LLVM Github Monorepo https://reviews.llvm.org/D152416 Files: clang/include/clang/Driver/Options