It's not intended, but r316661 indeed fixed http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15 ;)
On Thu, Oct 26, 2017 at 3:16 PM Eric Liu <ioe...@google.com> wrote: > Hi Maxim, > > Thanks for the email! I'll look into this right now. > > Thanks, > Eric > > On Thu, Oct 26, 2017 at 3:11 PM Maxim Kuvyrkov <maxim.kuvyr...@linaro.org> > wrote: > >> Hi Eric, >> >> This seems to have broken clang-cmake-thumbv7-a15 buildbot [*]. Would >> you please investigate? >> >> Or did you fix this while I was writing this email in 316661 :-) ? >> >> Thank you, >> >> [*] >> http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15/builds/12511 >> >> -- >> Maxim Kuvyrkov >> www.linaro.org >> >> >> >> > On Oct 26, 2017, at 1:38 PM, Eric Liu via cfe-commits < >> cfe-commits@lists.llvm.org> wrote: >> > >> > Author: ioeric >> > Date: Thu Oct 26 03:38:14 2017 >> > New Revision: 316653 >> > >> > URL: http://llvm.org/viewvc/llvm-project?rev=316653&view=rev >> > Log: >> > [Tooling] A new framework for executing clang frontend actions. >> > >> > Summary: >> > This defines a `clang::tooling::ToolExecutor` interface that can be >> extended to support different execution plans including standalone >> execution on a given set of TUs or parallel execution on all TUs in a >> codebase. >> > >> > In order to enable multiprocessing execution, tool actions are expected >> to output result into a `ToolResults` interface provided by executors. The >> `ToolResults` interface abstracts how results are stored e.g. in-memory for >> standalone executions or on-disk for large-scale execution. >> > >> > New executors can be registered as `ToolExecutorPlugin`s via the >> `ToolExecutorPluginRegistry`. CLI tools can use >> `createExecutorFromCommandLineArgs` to create a specific registered >> executor according to the command-line arguments. >> > >> > This patch also implements `StandaloneToolExecutor` which has the same >> behavior as the current `ClangTool` interface, i.e. execute frontend >> actions on a given set of TUs. At this point, it's simply a wrapper around >> `ClangTool` at this point. >> > >> > This is still experimental but expected to replace the existing >> `ClangTool` interface so that specific tools would not need to worry about >> execution. >> > >> > Reviewers: klimek, arphaman, hokein, sammccall >> > >> > Reviewed By: klimek >> > >> > Subscribers: cfe-commits, djasper, mgorny, omtcyfz >> > >> > Differential Revision: https://reviews.llvm.org/D34272 >> > >> > Added: >> > cfe/trunk/include/clang/Tooling/Execution.h >> > cfe/trunk/include/clang/Tooling/StandaloneExecution.h >> > cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h >> > cfe/trunk/lib/Tooling/Execution.cpp >> > cfe/trunk/lib/Tooling/StandaloneExecution.cpp >> > cfe/trunk/unittests/Tooling/ExecutionTest.cpp >> > Modified: >> > cfe/trunk/include/clang/Tooling/CommonOptionsParser.h >> > cfe/trunk/include/clang/Tooling/Tooling.h >> > cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp >> > cfe/trunk/lib/Tooling/CMakeLists.txt >> > cfe/trunk/lib/Tooling/CommonOptionsParser.cpp >> > cfe/trunk/lib/Tooling/Tooling.cpp >> > cfe/trunk/unittests/Tooling/CMakeLists.txt >> > >> > Modified: cfe/trunk/include/clang/Tooling/CommonOptionsParser.h >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/CommonOptionsParser.h?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/include/clang/Tooling/CommonOptionsParser.h (original) >> > +++ cfe/trunk/include/clang/Tooling/CommonOptionsParser.h Thu Oct 26 >> 03:38:14 2017 >> > @@ -109,6 +109,10 @@ public: >> > return SourcePathList; >> > } >> > >> > + /// Returns the argument adjuster calculated from "--extra-arg" and >> > + //"--extra-arg-before" options. >> > + ArgumentsAdjuster getArgumentsAdjuster() { return Adjuster; } >> > + >> > static const char *const HelpMessage; >> > >> > private: >> > @@ -121,6 +125,7 @@ private: >> > >> > std::unique_ptr<CompilationDatabase> Compilations; >> > std::vector<std::string> SourcePathList; >> > + ArgumentsAdjuster Adjuster; >> > }; >> > >> > class ArgumentsAdjustingCompilations : public CompilationDatabase { >> > >> > Added: cfe/trunk/include/clang/Tooling/Execution.h >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Execution.h?rev=316653&view=auto >> > >> ============================================================================== >> > --- cfe/trunk/include/clang/Tooling/Execution.h (added) >> > +++ cfe/trunk/include/clang/Tooling/Execution.h Thu Oct 26 03:38:14 2017 >> > @@ -0,0 +1,168 @@ >> > +//===--- Execution.h - Executing clang frontend actions -*- C++ >> ---------*-===// >> > +// >> > +// The LLVM Compiler Infrastructure >> > +// >> > +// This file is distributed under the University of Illinois Open >> Source >> > +// License. See LICENSE.TXT for details. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > +// >> > +// This file defines framework for executing clang frontend actions. >> > +// >> > +// The framework can be extended to support different execution plans >> including >> > +// standalone execution on the given TUs or parallel execution on all >> TUs in >> > +// the codebase. >> > +// >> > +// In order to enable multiprocessing execution, tool actions are >> expected to >> > +// output result into the ToolResults provided by the executor. The >> > +// `ToolResults` is an interface that abstracts how results are >> stored e.g. >> > +// in-memory for standalone execution or on-disk for large-scale >> execution. >> > +// >> > +// New executors can be registered as ToolExecutorPlugins via the >> > +// `ToolExecutorPluginRegistry`. CLI tools can use >> > +// `createExecutorFromCommandLineArgs` to create a specific >> registered executor >> > +// according to the command-line arguments. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > + >> > +#ifndef LLVM_CLANG_TOOLING_EXECUTION_H >> > +#define LLVM_CLANG_TOOLING_EXECUTION_H >> > + >> > +#include "clang/Tooling/CommonOptionsParser.h" >> > +#include "clang/Tooling/Tooling.h" >> > +#include "llvm/Support/Error.h" >> > +#include "llvm/Support/Registry.h" >> > + >> > +namespace clang { >> > +namespace tooling { >> > + >> > +/// \brief An abstraction for the result of a tool execution. For >> example, the >> > +/// underlying result can be in-memory or on-disk. >> > +/// >> > +/// Results should be string key-value pairs. For example, a >> refactoring tool >> > +/// can use source location as key and a replacement in YAML format as >> value. >> > +class ToolResults { >> > +public: >> > + virtual ~ToolResults() = default; >> > + virtual void addResult(StringRef Key, StringRef Value) = 0; >> > + virtual std::vector<std::pair<std::string, std::string>> >> AllKVResults() = 0; >> > + virtual void forEachResult( >> > + llvm::function_ref<void(StringRef Key, StringRef Value)> >> Callback) = 0; >> > +}; >> > + >> > +class InMemoryToolResults : public ToolResults { >> > +public: >> > + void addResult(StringRef Key, StringRef Value) override; >> > + std::vector<std::pair<std::string, std::string>> AllKVResults() >> override; >> > + void forEachResult(llvm::function_ref<void(StringRef Key, StringRef >> Value)> >> > + Callback) override; >> > + >> > +private: >> > + std::vector<std::pair<std::string, std::string>> KVResults; >> > +}; >> > + >> > +/// \brief The context of an execution, including the information about >> > +/// compilation and results. >> > +class ExecutionContext { >> > +public: >> > + virtual ~ExecutionContext() {} >> > + >> > + /// \brief Initializes a context. This does not take ownership of >> `Results`. >> > + explicit ExecutionContext(ToolResults *Results) : Results(Results) {} >> > + >> > + /// \brief Adds a KV pair to the result container of this execution. >> > + void reportResult(StringRef Key, StringRef Value); >> > + >> > + // Returns the source control system's revision number if applicable. >> > + // Otherwise returns an empty string. >> > + virtual std::string getRevision() { return ""; } >> > + >> > + // Returns the corpus being analyzed, e.g. "llvm" for the LLVM >> codebase, if >> > + // applicable. >> > + virtual std::string getCorpus() { return ""; } >> > + >> > + // Returns the currently processed compilation unit if available. >> > + virtual std::string getCurrentCompilationUnit() { return ""; } >> > + >> > +private: >> > + ToolResults *Results; >> > +}; >> > + >> > +/// \brief Interface for executing clang frontend actions. >> > +/// >> > +/// This can be extended to support running tool actions in different >> > +/// execution mode, e.g. on a specific set of TUs or many TUs in >> parallel. >> > +/// >> > +/// New executors can be registered as ToolExecutorPlugins via the >> > +/// `ToolExecutorPluginRegistry`. CLI tools can use >> > +/// `createExecutorFromCommandLineArgs` to create a specific >> registered >> > +/// executor according to the command-line arguments. >> > +class ToolExecutor { >> > +public: >> > + virtual ~ToolExecutor() {} >> > + >> > + /// \brief Returns the name of a specific executor. >> > + virtual StringRef getExecutorName() const = 0; >> > + >> > + /// \brief Executes each action with a corresponding arguments >> adjuster. >> > + virtual llvm::Error >> > + execute(llvm::ArrayRef< >> > + std::pair<std::unique_ptr<FrontendActionFactory>, >> ArgumentsAdjuster>> >> > + Actions) = 0; >> > + >> > + /// \brief Convenient functions for the above `execute`. >> > + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action); >> > + /// Executes an action with an argument adjuster. >> > + llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action, >> > + ArgumentsAdjuster Adjuster); >> > + >> > + /// \brief Returns a reference to the execution context. >> > + /// >> > + /// This should be passed to tool callbacks, and tool callbacks >> should report >> > + /// results via the returned context. >> > + virtual ExecutionContext *getExecutionContext() = 0; >> > + >> > + /// \brief Returns a reference to the result container. >> > + /// >> > + /// NOTE: This should only be used after the execution finishes. Tool >> > + /// callbacks should report results via `ExecutionContext` instead. >> > + virtual ToolResults *getToolResults() = 0; >> > + >> > + /// \brief Map a virtual file to be used while running the tool. >> > + /// >> > + /// \param FilePath The path at which the content will be mapped. >> > + /// \param Content A buffer of the file's content. >> > + virtual void mapVirtualFile(StringRef FilePath, StringRef Content) = >> 0; >> > +}; >> > + >> > +/// \brief Interface for factories that create specific executors. >> This is also >> > +/// used as a plugin to be registered into ToolExecutorPluginRegistry. >> > +class ToolExecutorPlugin { >> > +public: >> > + virtual ~ToolExecutorPlugin() {} >> > + >> > + /// \brief Create an `ToolExecutor`. >> > + /// >> > + /// `OptionsParser` can be consumed (e.g. moved) if the creation >> succeeds. >> > + virtual llvm::Expected<std::unique_ptr<ToolExecutor>> >> > + create(CommonOptionsParser &OptionsParser) = 0; >> > +}; >> > + >> > +/// \brief This creates a ToolExecutor that is in the global registry >> based on >> > +/// commandline arguments. >> > +/// >> > +/// This picks the right executor based on the `--executor` option. >> This parses >> > +/// the commandline arguments with `CommonOptionsParser`, so caller >> does not >> > +/// need to parse again. >> > +/// >> > +/// By default, this creates a `StandaloneToolExecutor` ("standalone") >> if >> > +/// `--executor` is not provided. >> > +llvm::Expected<std::unique_ptr<ToolExecutor>> >> > +createExecutorFromCommandLineArgs(int &argc, const char **argv, >> > + llvm::cl::OptionCategory &Category, >> > + const char *Overview = nullptr); >> > + >> > +} // end namespace tooling >> > +} // end namespace clang >> > + >> > +#endif // LLVM_CLANG_TOOLING_EXECUTION_H >> > >> > Added: cfe/trunk/include/clang/Tooling/StandaloneExecution.h >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/StandaloneExecution.h?rev=316653&view=auto >> > >> ============================================================================== >> > --- cfe/trunk/include/clang/Tooling/StandaloneExecution.h (added) >> > +++ cfe/trunk/include/clang/Tooling/StandaloneExecution.h Thu Oct 26 >> 03:38:14 2017 >> > @@ -0,0 +1,97 @@ >> > +//===--- StandaloneExecution.h - Standalone execution. -*- C++ >> ----------*-===// >> > +// >> > +// The LLVM Compiler Infrastructure >> > +// >> > +// This file is distributed under the University of Illinois Open >> Source >> > +// License. See LICENSE.TXT for details. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > +// >> > +// This file defines standalone execution of clang tools. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > + >> > +#ifndef LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H >> > +#define LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H >> > + >> > +#include "clang/Tooling/ArgumentsAdjusters.h" >> > +#include "clang/Tooling/Execution.h" >> > + >> > +namespace clang { >> > +namespace tooling { >> > + >> > +/// \brief A standalone executor that runs FrontendActions on a given >> set of >> > +/// TUs in sequence. >> > +/// >> > +/// By default, this executor uses the following arguments adjusters >> (as defined >> > +/// in `clang/Tooling/ArgumentsAdjusters.h`): >> > +/// - `getClangStripOutputAdjuster()` >> > +/// - `getClangSyntaxOnlyAdjuster()` >> > +/// - `getClangStripDependencyFileAdjuster()` >> > +class StandaloneToolExecutor : public ToolExecutor { >> > +public: >> > + static const char *ExecutorName; >> > + >> > + /// \brief Init with \p CompilationDatabase and the paths of all >> files to be >> > + /// proccessed. >> > + StandaloneToolExecutor( >> > + const CompilationDatabase &Compilations, >> > + llvm::ArrayRef<std::string> SourcePaths, >> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps = >> > + std::make_shared<PCHContainerOperations>()); >> > + >> > + /// \brief Init with \p CommonOptionsParser. This is expected to be >> used by >> > + /// `createExecutorFromCommandLineArgs` based on commandline options. >> > + /// >> > + /// The executor takes ownership of \p Options. >> > + StandaloneToolExecutor( >> > + CommonOptionsParser Options, >> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps = >> > + std::make_shared<PCHContainerOperations>()); >> > + >> > + StringRef getExecutorName() const override { return ExecutorName; } >> > + >> > + using ToolExecutor::execute; >> > + >> > + llvm::Error >> > + execute(llvm::ArrayRef< >> > + std::pair<std::unique_ptr<FrontendActionFactory>, >> ArgumentsAdjuster>> >> > + Actions) override; >> > + >> > + /// \brief Set a \c DiagnosticConsumer to use during parsing. >> > + void setDiagnosticConsumer(DiagnosticConsumer *DiagConsumer) { >> > + Tool.setDiagnosticConsumer(DiagConsumer); >> > + } >> > + >> > + ExecutionContext *getExecutionContext() override { return &Context; >> }; >> > + >> > + ToolResults *getToolResults() override { return &Results; } >> > + >> > + llvm::ArrayRef<std::string> getSourcePaths() const { >> > + return Tool.getSourcePaths(); >> > + } >> > + >> > + void mapVirtualFile(StringRef FilePath, StringRef Content) override { >> > + Tool.mapVirtualFile(FilePath, Content); >> > + } >> > + >> > + /// \brief Returns the file manager used in the tool. >> > + /// >> > + /// The file manager is shared between all translation units. >> > + FileManager &getFiles() { return Tool.getFiles(); } >> > + >> > +private: >> > + // Used to store the parser when the executor is initialized with >> parser. >> > + llvm::Optional<CommonOptionsParser> OptionsParser; >> > + // FIXME: The standalone executor is currently just a wrapper of >> `ClangTool`. >> > + // Merge `ClangTool` implementation into the this. >> > + ClangTool Tool; >> > + ExecutionContext Context; >> > + InMemoryToolResults Results; >> > + ArgumentsAdjuster ArgsAdjuster; >> > +}; >> > + >> > +} // end namespace tooling >> > +} // end namespace clang >> > + >> > +#endif // LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H >> > >> > Added: cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h?rev=316653&view=auto >> > >> ============================================================================== >> > --- cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h (added) >> > +++ cfe/trunk/include/clang/Tooling/ToolExecutorPluginRegistry.h Thu >> Oct 26 03:38:14 2017 >> > @@ -0,0 +1,24 @@ >> > +//===--- ToolExecutorPluginRegistry.h - -------------------------*- >> C++ -*-===// >> > +// >> > +// The LLVM Compiler Infrastructure >> > +// >> > +// This file is distributed under the University of Illinois Open >> Source >> > +// License. See LICENSE.TXT for details. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > + >> > +#ifndef LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H >> > +#define LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H >> > + >> > +#include "clang/Tooling/Execution.h" >> > +#include "llvm/Support/Registry.h" >> > + >> > +namespace clang { >> > +namespace tooling { >> > + >> > +typedef llvm::Registry<ToolExecutorPlugin> ToolExecutorPluginRegistry; >> > + >> > +} // end namespace tooling >> > +} // end namespace clang >> > + >> > +#endif // LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H >> > >> > Modified: cfe/trunk/include/clang/Tooling/Tooling.h >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Tooling.h?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/include/clang/Tooling/Tooling.h (original) >> > +++ cfe/trunk/include/clang/Tooling/Tooling.h Thu Oct 26 03:38:14 2017 >> > @@ -31,12 +31,12 @@ >> > #define LLVM_CLANG_TOOLING_TOOLING_H >> > >> > #include "clang/AST/ASTConsumer.h" >> > -#include "clang/Frontend/PCHContainerOperations.h" >> > #include "clang/Basic/Diagnostic.h" >> > #include "clang/Basic/FileManager.h" >> > #include "clang/Basic/LLVM.h" >> > #include "clang/Driver/Util.h" >> > #include "clang/Frontend/FrontendAction.h" >> > +#include "clang/Frontend/PCHContainerOperations.h" >> > #include "clang/Lex/ModuleLoader.h" >> > #include "clang/Tooling/ArgumentsAdjusters.h" >> > #include "clang/Tooling/CompilationDatabase.h" >> > @@ -337,7 +337,9 @@ class ClangTool { >> > /// The file manager is shared between all translation units. >> > FileManager &getFiles() { return *Files; } >> > >> > - private: >> > + llvm::ArrayRef<std::string> getSourcePaths() const { return >> SourcePaths; } >> > + >> > +private: >> > const CompilationDatabase &Compilations; >> > std::vector<std::string> SourcePaths; >> > std::shared_ptr<PCHContainerOperations> PCHContainerOps; >> > >> > Modified: cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp (original) >> > +++ cfe/trunk/lib/Tooling/ArgumentsAdjusters.cpp Thu Oct 26 03:38:14 >> 2017 >> > @@ -96,6 +96,10 @@ ArgumentsAdjuster getInsertArgumentAdjus >> > >> > ArgumentsAdjuster combineAdjusters(ArgumentsAdjuster First, >> > ArgumentsAdjuster Second) { >> > + if (!First) >> > + return Second; >> > + if (!Second) >> > + return First; >> > return [First, Second](const CommandLineArguments &Args, StringRef >> File) { >> > return Second(First(Args, File), File); >> > }; >> > >> > Modified: cfe/trunk/lib/Tooling/CMakeLists.txt >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/lib/Tooling/CMakeLists.txt (original) >> > +++ cfe/trunk/lib/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017 >> > @@ -11,11 +11,13 @@ add_clang_library(clangTooling >> > ArgumentsAdjusters.cpp >> > CommonOptionsParser.cpp >> > CompilationDatabase.cpp >> > + Execution.cpp >> > FileMatchTrie.cpp >> > FixIt.cpp >> > JSONCompilationDatabase.cpp >> > Refactoring.cpp >> > RefactoringCallbacks.cpp >> > + StandaloneExecution.cpp >> > Tooling.cpp >> > >> > DEPENDS >> > >> > Modified: cfe/trunk/lib/Tooling/CommonOptionsParser.cpp >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/CommonOptionsParser.cpp?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/lib/Tooling/CommonOptionsParser.cpp (original) >> > +++ cfe/trunk/lib/Tooling/CommonOptionsParser.cpp Thu Oct 26 03:38:14 >> 2017 >> > @@ -147,10 +147,12 @@ llvm::Error CommonOptionsParser::init( >> > auto AdjustingCompilations = >> > llvm::make_unique<ArgumentsAdjustingCompilations>( >> > std::move(Compilations)); >> > - AdjustingCompilations->appendArgumentsAdjuster( >> > - getInsertArgumentAdjuster(ArgsBefore, >> ArgumentInsertPosition::BEGIN)); >> > - AdjustingCompilations->appendArgumentsAdjuster( >> > + Adjuster = >> > + getInsertArgumentAdjuster(ArgsBefore, >> ArgumentInsertPosition::BEGIN); >> > + Adjuster = combineAdjusters( >> > + std::move(Adjuster), >> > getInsertArgumentAdjuster(ArgsAfter, >> ArgumentInsertPosition::END)); >> > + AdjustingCompilations->appendArgumentsAdjuster(Adjuster); >> > Compilations = std::move(AdjustingCompilations); >> > return llvm::Error::success(); >> > } >> > >> > Added: cfe/trunk/lib/Tooling/Execution.cpp >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Execution.cpp?rev=316653&view=auto >> > >> ============================================================================== >> > --- cfe/trunk/lib/Tooling/Execution.cpp (added) >> > +++ cfe/trunk/lib/Tooling/Execution.cpp Thu Oct 26 03:38:14 2017 >> > @@ -0,0 +1,89 @@ >> > +//===- lib/Tooling/Execution.cpp - Implements tool execution >> framework. ---===// >> > +// >> > +// The LLVM Compiler Infrastructure >> > +// >> > +// This file is distributed under the University of Illinois Open >> Source >> > +// License. See LICENSE.TXT for details. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > + >> > +#include "clang/Tooling/Execution.h" >> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h" >> > +#include "clang/Tooling/Tooling.h" >> > + >> > +LLVM_INSTANTIATE_REGISTRY(clang::tooling::ToolExecutorPluginRegistry) >> > + >> > +namespace clang { >> > +namespace tooling { >> > + >> > +static llvm::cl::opt<std::string> >> > + ExecutorName("executor", llvm::cl::desc("The name of the executor >> to use."), >> > + llvm::cl::init("standalone")); >> > + >> > +void InMemoryToolResults::addResult(StringRef Key, StringRef Value) { >> > + KVResults.push_back({Key.str(), Value.str()}); >> > +} >> > + >> > +std::vector<std::pair<std::string, std::string>> >> > +InMemoryToolResults::AllKVResults() { >> > + return KVResults; >> > +} >> > + >> > +void InMemoryToolResults::forEachResult( >> > + llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) >> { >> > + for (const auto &KV : KVResults) { >> > + Callback(KV.first, KV.second); >> > + } >> > +} >> > + >> > +void ExecutionContext::reportResult(StringRef Key, StringRef Value) { >> > + Results->addResult(Key, Value); >> > +} >> > + >> > +llvm::Error >> > +ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action) { >> > + return execute(std::move(Action), ArgumentsAdjuster()); >> > +} >> > + >> > +llvm::Error >> ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action, >> > + ArgumentsAdjuster Adjuster) { >> > + std::vector< >> > + std::pair<std::unique_ptr<FrontendActionFactory>, >> ArgumentsAdjuster>> >> > + Actions; >> > + Actions.emplace_back(std::move(Action), std::move(Adjuster)); >> > + return execute(Actions); >> > +} >> > + >> > +llvm::Expected<std::unique_ptr<ToolExecutor>> >> > +createExecutorFromCommandLineArgs(int &argc, const char **argv, >> > + llvm::cl::OptionCategory &Category, >> > + const char *Overview) { >> > + auto OptionsParser = >> > + CommonOptionsParser::create(argc, argv, Category, >> llvm::cl::ZeroOrMore, >> > + /*Overview=*/nullptr); >> > + if (!OptionsParser) >> > + return OptionsParser.takeError(); >> > + for (auto I = ToolExecutorPluginRegistry::begin(), >> > + E = ToolExecutorPluginRegistry::end(); >> > + I != E; ++I) { >> > + if (I->getName() != ExecutorName) { >> > + continue; >> > + } >> > + std::unique_ptr<ToolExecutorPlugin> Plugin(I->instantiate()); >> > + llvm::Expected<std::unique_ptr<ToolExecutor>> Executor = >> > + Plugin->create(*OptionsParser); >> > + if (!Executor) { >> > + return llvm::make_error<llvm::StringError>( >> > + llvm::Twine("Failed to create '") + I->getName() + >> > + "': " + llvm::toString(Executor.takeError()) + "\n", >> > + llvm::inconvertibleErrorCode()); >> > + } >> > + return std::move(*Executor); >> > + } >> > + return llvm::make_error<llvm::StringError>( >> > + llvm::Twine("Executor \"") + ExecutorName + "\" is not >> registered.", >> > + llvm::inconvertibleErrorCode()); >> > +} >> > + >> > +} // end namespace tooling >> > +} // end namespace clang >> > >> > Added: cfe/trunk/lib/Tooling/StandaloneExecution.cpp >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/StandaloneExecution.cpp?rev=316653&view=auto >> > >> ============================================================================== >> > --- cfe/trunk/lib/Tooling/StandaloneExecution.cpp (added) >> > +++ cfe/trunk/lib/Tooling/StandaloneExecution.cpp Thu Oct 26 03:38:14 >> 2017 >> > @@ -0,0 +1,91 @@ >> > +//===- lib/Tooling/Execution.cpp - Standalone clang action execution. >> -----===// >> > +// >> > +// The LLVM Compiler Infrastructure >> > +// >> > +// This file is distributed under the University of Illinois Open >> Source >> > +// License. See LICENSE.TXT for details. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > + >> > +#include "clang/Tooling/StandaloneExecution.h" >> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h" >> > + >> > +namespace clang { >> > +namespace tooling { >> > + >> > +static llvm::Error make_string_error(const llvm::Twine &Message) { >> > + return llvm::make_error<llvm::StringError>(Message, >> > + >> llvm::inconvertibleErrorCode()); >> > +} >> > + >> > +const char *StandaloneToolExecutor::ExecutorName = >> "StandaloneToolExecutor"; >> > + >> > +static ArgumentsAdjuster getDefaultArgumentsAdjusters() { >> > + return combineAdjusters( >> > + getClangStripOutputAdjuster(), >> > + combineAdjusters(getClangSyntaxOnlyAdjuster(), >> > + getClangStripDependencyFileAdjuster())); >> > +} >> > + >> > +StandaloneToolExecutor::StandaloneToolExecutor( >> > + const CompilationDatabase &Compilations, >> > + llvm::ArrayRef<std::string> SourcePaths, >> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps) >> > + : Tool(Compilations, SourcePaths), Context(&Results), >> > + ArgsAdjuster(getDefaultArgumentsAdjusters()) { >> > + // Use self-defined default argument adjusters instead of the default >> > + // adjusters that come with the old `ClangTool`. >> > + Tool.clearArgumentsAdjusters(); >> > +} >> > + >> > +StandaloneToolExecutor::StandaloneToolExecutor( >> > + CommonOptionsParser Options, >> > + std::shared_ptr<PCHContainerOperations> PCHContainerOps) >> > + : OptionsParser(std::move(Options)), >> > + Tool(OptionsParser->getCompilations(), >> OptionsParser->getSourcePathList(), >> > + PCHContainerOps), >> > + Context(&Results), ArgsAdjuster(getDefaultArgumentsAdjusters()) { >> > + Tool.clearArgumentsAdjusters(); >> > +} >> > + >> > +llvm::Error StandaloneToolExecutor::execute( >> > + llvm::ArrayRef< >> > + std::pair<std::unique_ptr<FrontendActionFactory>, >> ArgumentsAdjuster>> >> > + Actions) { >> > + if (Actions.empty()) >> > + return make_string_error("No action to execute."); >> > + >> > + if (Actions.size() != 1) >> > + return make_string_error( >> > + "Only support executing exactly 1 action at this point."); >> > + >> > + auto &Action = Actions.front(); >> > + Tool.appendArgumentsAdjuster(Action.second); >> > + Tool.appendArgumentsAdjuster(ArgsAdjuster); >> > + if (int Ret = Tool.run(Action.first.get())) >> > + return make_string_error("Failed to run action."); >> > + >> > + return llvm::Error::success(); >> > +} >> > + >> > +class StandaloneToolExecutorPlugin : public ToolExecutorPlugin { >> > +public: >> > + llvm::Expected<std::unique_ptr<ToolExecutor>> >> > + create(CommonOptionsParser &OptionsParser) override { >> > + if (OptionsParser.getSourcePathList().empty()) >> > + return make_string_error( >> > + "[StandaloneToolExecutorPlugin] No positional argument >> found."); >> > + return >> llvm::make_unique<StandaloneToolExecutor>(std::move(OptionsParser)); >> > + } >> > +}; >> > + >> > +// This anchor is used to force the linker to link in the generated >> object file >> > +// and thus register the plugin. >> > +volatile int ToolExecutorPluginAnchorSource = 0; >> > + >> > +static ToolExecutorPluginRegistry::Add<StandaloneToolExecutorPlugin> >> > + X("standalone", "Runs FrontendActions on a set of files provided " >> > + "via positional arguments."); >> > + >> > +} // end namespace tooling >> > +} // end namespace clang >> > >> > Modified: cfe/trunk/lib/Tooling/Tooling.cpp >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Tooling.cpp?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/lib/Tooling/Tooling.cpp (original) >> > +++ cfe/trunk/lib/Tooling/Tooling.cpp Thu Oct 26 03:38:14 2017 >> > @@ -29,6 +29,7 @@ >> > #include "llvm/Config/llvm-config.h" >> > #include "llvm/Option/ArgList.h" >> > #include "llvm/Option/Option.h" >> > +#include "llvm/Support/CommandLine.h" >> > #include "llvm/Support/Debug.h" >> > #include "llvm/Support/FileSystem.h" >> > #include "llvm/Support/Host.h" >> > @@ -347,11 +348,7 @@ void ClangTool::mapVirtualFile(StringRef >> > } >> > >> > void ClangTool::appendArgumentsAdjuster(ArgumentsAdjuster Adjuster) { >> > - if (ArgsAdjuster) >> > - ArgsAdjuster = >> > - combineAdjusters(std::move(ArgsAdjuster), std::move(Adjuster)); >> > - else >> > - ArgsAdjuster = std::move(Adjuster); >> > + ArgsAdjuster = combineAdjusters(std::move(ArgsAdjuster), >> std::move(Adjuster)); >> > } >> > >> > void ClangTool::clearArgumentsAdjusters() { >> > >> > Modified: cfe/trunk/unittests/Tooling/CMakeLists.txt >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/CMakeLists.txt?rev=316653&r1=316652&r2=316653&view=diff >> > >> ============================================================================== >> > --- cfe/trunk/unittests/Tooling/CMakeLists.txt (original) >> > +++ cfe/trunk/unittests/Tooling/CMakeLists.txt Thu Oct 26 03:38:14 2017 >> > @@ -16,6 +16,7 @@ add_clang_unittest(ToolingTests >> > CommentHandlerTest.cpp >> > CompilationDatabaseTest.cpp >> > DiagnosticsYamlTest.cpp >> > + ExecutionTest.cpp >> > FixItTest.cpp >> > LexicallyOrderedRecursiveASTVisitorTest.cpp >> > LookupTest.cpp >> > >> > Added: cfe/trunk/unittests/Tooling/ExecutionTest.cpp >> > URL: >> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/ExecutionTest.cpp?rev=316653&view=auto >> > >> ============================================================================== >> > --- cfe/trunk/unittests/Tooling/ExecutionTest.cpp (added) >> > +++ cfe/trunk/unittests/Tooling/ExecutionTest.cpp Thu Oct 26 03:38:14 >> 2017 >> > @@ -0,0 +1,228 @@ >> > +//===- unittest/Tooling/ExecutionTest.cpp - Tool execution tests. >> --------===// >> > +// >> > +// The LLVM Compiler Infrastructure >> > +// >> > +// This file is distributed under the University of Illinois Open >> Source >> > +// License. See LICENSE.TXT for details. >> > +// >> > >> +//===----------------------------------------------------------------------===// >> > + >> > +#include "clang/AST/ASTConsumer.h" >> > +#include "clang/AST/DeclCXX.h" >> > +#include "clang/AST/RecursiveASTVisitor.h" >> > +#include "clang/Frontend/ASTUnit.h" >> > +#include "clang/Frontend/FrontendAction.h" >> > +#include "clang/Frontend/FrontendActions.h" >> > +#include "clang/Tooling/CompilationDatabase.h" >> > +#include "clang/Tooling/Execution.h" >> > +#include "clang/Tooling/StandaloneExecution.h" >> > +#include "clang/Tooling/ToolExecutorPluginRegistry.h" >> > +#include "clang/Tooling/Tooling.h" >> > +#include "gtest/gtest.h" >> > +#include <algorithm> >> > +#include <string> >> > + >> > +namespace clang { >> > +namespace tooling { >> > + >> > +namespace { >> > + >> > +// This traverses the AST and outputs function name as key and "1" as >> value for >> > +// each function declaration. >> > +class ASTConsumerWithResult >> > + : public ASTConsumer, >> > + public RecursiveASTVisitor<ASTConsumerWithResult> { >> > +public: >> > + using ASTVisitor = RecursiveASTVisitor<ASTConsumerWithResult>; >> > + >> > + explicit ASTConsumerWithResult(ExecutionContext *Context) : >> Context(Context) { >> > + assert(Context != nullptr); >> > + } >> > + >> > + void HandleTranslationUnit(clang::ASTContext &Context) override { >> > + TraverseDecl(Context.getTranslationUnitDecl()); >> > + } >> > + >> > + bool TraverseFunctionDecl(clang::FunctionDecl *Decl) { >> > + Context->reportResult(Decl->getNameAsString(), "1"); >> > + return ASTVisitor::TraverseFunctionDecl(Decl); >> > + } >> > + >> > +private: >> > + ExecutionContext *const Context; >> > +}; >> > + >> > +class ReportResultAction : public ASTFrontendAction { >> > +public: >> > + explicit ReportResultAction(ExecutionContext *Context) : >> Context(Context) { >> > + assert(Context != nullptr); >> > + } >> > + >> > +protected: >> > + std::unique_ptr<clang::ASTConsumer> >> > + CreateASTConsumer(clang::CompilerInstance &compiler, >> > + StringRef /* dummy */) override { >> > + std::unique_ptr<clang::ASTConsumer> ast_consumer{ >> > + new ASTConsumerWithResult(Context)}; >> > + return ast_consumer; >> > + } >> > + >> > +private: >> > + ExecutionContext *const Context; >> > +}; >> > + >> > +class ReportResultActionFactory : public FrontendActionFactory { >> > +public: >> > + ReportResultActionFactory(ExecutionContext *Context) : >> Context(Context) {} >> > + FrontendAction *create() override { return new >> ReportResultAction(Context); } >> > + >> > +private: >> > + ExecutionContext *const Context; >> > +}; >> > + >> > +} // namespace >> > + >> > +class TestToolExecutor : public ToolExecutor { >> > +public: >> > + static const char *ExecutorName; >> > + >> > + TestToolExecutor(CommonOptionsParser Options) >> > + : OptionsParser(std::move(Options)) {} >> > + >> > + StringRef getExecutorName() const override { return ExecutorName; } >> > + >> > + llvm::Error >> > + >> execute(llvm::ArrayRef<std::pair<std::unique_ptr<FrontendActionFactory>, >> > + ArgumentsAdjuster>>) override { >> > + return llvm::Error::success(); >> > + } >> > + >> > + ExecutionContext *getExecutionContext() override { return nullptr; }; >> > + >> > + ToolResults *getToolResults() override { return nullptr; } >> > + >> > + llvm::ArrayRef<std::string> getSourcePaths() const { >> > + return OptionsParser.getSourcePathList(); >> > + } >> > + >> > + void mapVirtualFile(StringRef FilePath, StringRef Content) override { >> > + VFS[FilePath] = Content; >> > + } >> > + >> > +private: >> > + CommonOptionsParser OptionsParser; >> > + std::string SourcePaths; >> > + std::map<std::string, std::string> VFS; >> > +}; >> > + >> > +const char *TestToolExecutor::ExecutorName = "test-executor"; >> > + >> > +class TestToolExecutorPlugin : public ToolExecutorPlugin { >> > +public: >> > + llvm::Expected<std::unique_ptr<ToolExecutor>> >> > + create(CommonOptionsParser &OptionsParser) override { >> > + return >> llvm::make_unique<TestToolExecutor>(std::move(OptionsParser)); >> > + } >> > +}; >> > + >> > +// This anchor is used to force the linker to link in the generated >> object file >> > +// and thus register the plugin. >> > +extern volatile int ToolExecutorPluginAnchorSource; >> > + >> > +static int LLVM_ATTRIBUTE_UNUSED TestToolExecutorPluginAnchorDest = >> > + ToolExecutorPluginAnchorSource; >> > + >> > +static ToolExecutorPluginRegistry::Add<TestToolExecutorPlugin> >> > + X("test-executor", "Plugin for TestToolExecutor."); >> > + >> > +llvm::cl::OptionCategory TestCategory("execution-test options"); >> > + >> > +TEST(CreateToolExecutorTest, FailedCreateExecutorUndefinedFlag) { >> > + std::vector<const char *> argv = {"prog", "--fake_flag_no_no_no", >> "f"}; >> > + int argc = argv.size(); >> > + auto Executor = >> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory); >> > + ASSERT_FALSE((bool)Executor); >> > + llvm::consumeError(Executor.takeError()); >> > +} >> > + >> > +TEST(CreateToolExecutorTest, RegisterFlagsBeforeReset) { >> > + llvm::cl::opt<std::string> BeforeReset( >> > + "before_reset", llvm::cl::desc("Defined before reset."), >> > + llvm::cl::init("")); >> > + >> > + llvm::cl::ResetAllOptionOccurrences(); >> > + >> > + std::vector<const char *> argv = {"prog", "--before_reset=set", "f"}; >> > + int argc = argv.size(); >> > + auto Executor = >> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory); >> > + ASSERT_TRUE((bool)Executor); >> > + EXPECT_EQ(BeforeReset, "set"); >> > + BeforeReset.removeArgument(); >> > +} >> > + >> > +TEST(CreateToolExecutorTest, CreateStandaloneToolExecutor) { >> > + std::vector<const char *> argv = {"prog", "standalone.cpp"}; >> > + int argc = argv.size(); >> > + auto Executor = >> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory); >> > + ASSERT_TRUE((bool)Executor); >> > + EXPECT_EQ(Executor->get()->getExecutorName(), >> > + StandaloneToolExecutor::ExecutorName); >> > +} >> > + >> > +TEST(CreateToolExecutorTest, CreateTestToolExecutor) { >> > + std::vector<const char *> argv = {"prog", "test.cpp", >> > + "--executor=test-executor"}; >> > + int argc = argv.size(); >> > + auto Executor = >> > + createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory); >> > + ASSERT_TRUE((bool)Executor); >> > + EXPECT_EQ(Executor->get()->getExecutorName(), >> TestToolExecutor::ExecutorName); >> > +} >> > + >> > +TEST(StandaloneToolTest, SynctaxOnlyActionOnSimpleCode) { >> > + FixedCompilationDatabase Compilations("/", >> std::vector<std::string>()); >> > + StandaloneToolExecutor Executor(Compilations, >> > + std::vector<std::string>(1, >> "/a.cc")); >> > + Executor.mapVirtualFile("/a.cc", "int x = 0;"); >> > + >> > + auto Err = >> Executor.execute(newFrontendActionFactory<SyntaxOnlyAction>(), >> > + getClangSyntaxOnlyAdjuster()); >> > + ASSERT_TRUE(!Err); >> > +} >> > + >> > +TEST(StandaloneToolTest, SimpleAction) { >> > + FixedCompilationDatabase Compilations("/", >> std::vector<std::string>()); >> > + StandaloneToolExecutor Executor(Compilations, >> > + std::vector<std::string>(1, >> "/a.cc")); >> > + Executor.mapVirtualFile("/a.cc", "int x = 0;"); >> > + >> > + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>( >> > + new ReportResultActionFactory(Executor.getExecutionContext()))); >> > + ASSERT_TRUE(!Err); >> > + auto KVs = Executor.getToolResults()->AllKVResults(); >> > + ASSERT_EQ(KVs.size(), 0u); >> > +} >> > + >> > +TEST(StandaloneToolTest, SimpleActionWithResult) { >> > + FixedCompilationDatabase Compilations("/", >> std::vector<std::string>()); >> > + StandaloneToolExecutor Executor(Compilations, >> > + std::vector<std::string>(1, >> "/a.cc")); >> > + Executor.mapVirtualFile("/a.cc", "int x = 0; void f() {}"); >> > + >> > + auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>( >> > + new ReportResultActionFactory(Executor.getExecutionContext()))); >> > + ASSERT_TRUE(!Err); >> > + auto KVs = Executor.getToolResults()->AllKVResults(); >> > + ASSERT_EQ(KVs.size(), 1u); >> > + EXPECT_EQ("f", KVs[0].first); >> > + EXPECT_EQ("1", KVs[0].second); >> > + >> > + Executor.getToolResults()->forEachResult( >> > + [](StringRef, StringRef Value) { EXPECT_EQ("1", Value); }); >> > +} >> > + >> > +} // end namespace tooling >> > +} // end namespace clang >> > >> > >> > _______________________________________________ >> > cfe-commits mailing list >> > cfe-commits@lists.llvm.org >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits >> >>
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
