Hi Ilya,
The unit test introduced by this commit hits an assertion failure on our bots.
Could you please take a look at the issue?
lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6733
<http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6733>
I see:
[ RUN ] ClangdVFSTest.Reparse
Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit, file
/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp,
line 58.
0 ClangdTests 0x0000000102fec2f8
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1 ClangdTests 0x0000000102fec9e6 SignalHandler(int) + 454
2 libsystem_platform.dylib 0x00007fff97ed352a _sigtramp + 26
3 libsystem_platform.dylib 000000000000000000 _sigtramp + 1746062064
4 libsystem_c.dylib 0x00007fff97d776df abort + 129
5 libsystem_c.dylib 0x00007fff97d3edd8 basename + 0
6 ClangdTests 0x00000001030c95dd
clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef,
std::__1::shared_ptr<clang::PCHContainerOperations>,
std::__1::vector<clang::tooling::CompileCommand,
std::__1::allocator<clang::tooling::CompileCommand> >,
llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) + 1837
7 ClangdTests 0x00000001030c824f
std::__1::__function::__func<clang::clangd::ClangdServer::addDocument(llvm::StringRef,
llvm::StringRef)::$_1,
std::__1::allocator<clang::clangd::ClangdServer::addDocument(llvm::StringRef,
llvm::StringRef)::$_1>, void ()>::operator()() + 975
8 ClangdTests 0x00000001030c6e03 void*
std::__1::__thread_proxy<std::__1::tuple<clang::clangd::ClangdScheduler::ClangdScheduler(bool)::$_0>
>(void*) + 579
9 libsystem_pthread.dylib 0x00007fff8e26099d _pthread_body + 131
10 libsystem_pthread.dylib 0x00007fff8e26091a _pthread_body + 0
11 libsystem_pthread.dylib 0x00007fff8e25e351 thread_start + 13
********************
FAIL: Extra Tools Unit Tests ::
clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange (13778 of 37955)
******************** TEST 'Extra Tools Unit Tests ::
clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange' FAILED
********************
Note: Google Test filter = ClangdVFSTest.ReparseOnHeaderChange
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ClangdVFSTest
[ RUN ] ClangdVFSTest.ReparseOnHeaderChange
Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit, file
/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp,
line 58.
0 ClangdTests 0x00000001005b72f8
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1 ClangdTests 0x00000001005b79e6 SignalHandler(int) + 454
2 libsystem_platform.dylib 0x00007fff97ed352a _sigtramp + 26
3 libsystem_platform.dylib 000000000000000000 _sigtramp + 1746062064
4 libsystem_c.dylib 0x00007fff97d776df abort + 129
5 libsystem_c.dylib 0x00007fff97d3edd8 basename + 0
6 ClangdTests 0x00000001006945dd
clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef,
std::__1::shared_ptr<clang::PCHContainerOperations>,
std::__1::vector<clang::tooling::CompileCommand,
std::__1::allocator<clang::tooling::CompileCommand> >,
llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) + 1837
7 ClangdTests 0x000000010069324f
std::__1::__function::__func<clang::clangd::ClangdServer::addDocument(llvm::StringRef,
llvm::StringRef)::$_1,
std::__1::allocator<clang::clangd::ClangdServer::addDocument(llvm::StringRef,
llvm::StringRef)::$_1>, void ()>::operator()() + 975
8 ClangdTests 0x0000000100691e03 void*
std::__1::__thread_proxy<std::__1::tuple<clang::clangd::ClangdScheduler::ClangdScheduler(bool)::$_0>
>(void*) + 579
9 libsystem_pthread.dylib 0x00007fff8e26099d _pthread_body + 131
10 libsystem_pthread.dylib 0x00007fff8e26091a _pthread_body + 0
11 libsystem_pthread.dylib 0x00007fff8e25e351 thread_start + 13
thanks,
vedant
> On May 26, 2017, at 5:26 AM, Ilya Biryukov via cfe-commits
> <cfe-commits@lists.llvm.org> wrote:
>
> Author: ibiryukov
> Date: Fri May 26 07:26:51 2017
> New Revision: 303977
>
> URL: http://llvm.org/viewvc/llvm-project?rev=303977&view=rev
> Log:
> [clangd] Allow to use vfs::FileSystem for file accesses.
>
> Summary:
> Custom vfs::FileSystem is currently used for unit tests.
> This revision depends on https://reviews.llvm.org/D33397.
>
> Reviewers: bkramer, krasimir
>
> Reviewed By: bkramer, krasimir
>
> Subscribers: klimek, cfe-commits, mgorny
>
> Differential Revision: https://reviews.llvm.org/D33416
>
> Added:
> clang-tools-extra/trunk/unittests/clangd/
> clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
> clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
> Modified:
> clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
> clang-tools-extra/trunk/clangd/ClangdServer.cpp
> clang-tools-extra/trunk/clangd/ClangdServer.h
> clang-tools-extra/trunk/clangd/ClangdUnit.cpp
> clang-tools-extra/trunk/clangd/ClangdUnit.h
> clang-tools-extra/trunk/clangd/ClangdUnitStore.h
> clang-tools-extra/trunk/unittests/CMakeLists.txt
>
> Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
> +++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Fri May 26 07:26:51
> 2017
> @@ -199,6 +199,7 @@ ClangdLSPServer::ClangdLSPServer(JSONOut
> : Out(Out),
> Server(llvm::make_unique<DirectoryBasedGlobalCompilationDatabase>(),
> llvm::make_unique<LSPDiagnosticsConsumer>(*this),
> + llvm::make_unique<RealFileSystemProvider>(),
> RunSynchronously) {}
>
> void ClangdLSPServer::run(std::istream &In) {
>
> Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
> +++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Fri May 26 07:26:51 2017
> @@ -58,6 +58,10 @@ Position clangd::offsetToPosition(String
> return {Lines, Cols};
> }
>
> +IntrusiveRefCntPtr<vfs::FileSystem> RealFileSystemProvider::getFileSystem() {
> + return vfs::getRealFileSystem();
> +}
> +
> ClangdScheduler::ClangdScheduler(bool RunSynchronously)
> : RunSynchronously(RunSynchronously) {
> if (RunSynchronously) {
> @@ -135,8 +139,10 @@ void ClangdScheduler::addToEnd(std::func
>
> ClangdServer::ClangdServer(std::unique_ptr<GlobalCompilationDatabase> CDB,
> std::unique_ptr<DiagnosticsConsumer> DiagConsumer,
> + std::unique_ptr<FileSystemProvider> FSProvider,
> bool RunSynchronously)
> : CDB(std::move(CDB)), DiagConsumer(std::move(DiagConsumer)),
> + FSProvider(std::move(FSProvider)),
> PCHs(std::make_shared<PCHContainerOperations>()),
> WorkScheduler(RunSynchronously) {}
>
> @@ -150,10 +156,11 @@ void ClangdServer::addDocument(PathRef F
>
> assert(FileContents.Draft &&
> "No contents inside a file that was scheduled for reparse");
> - Units.runOnUnit(
> - FileStr, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit const
> &Unit) {
> - DiagConsumer->onDiagnosticsReady(FileStr,
> Unit.getLocalDiagnostics());
> - });
> + Units.runOnUnit(FileStr, *FileContents.Draft, *CDB, PCHs,
> + FSProvider->getFileSystem(), [&](ClangdUnit const &Unit)
> {
> + DiagConsumer->onDiagnosticsReady(
> + FileStr, Unit.getLocalDiagnostics());
> + });
> });
> }
>
> @@ -168,15 +175,22 @@ void ClangdServer::removeDocument(PathRe
> });
> }
>
> +void ClangdServer::forceReparse(PathRef File) {
> + // The addDocument schedules the reparse even if the contents of the file
> + // never changed, so we just call it here.
> + addDocument(File, getDocument(File));
> +}
> +
> std::vector<CompletionItem> ClangdServer::codeComplete(PathRef File,
> Position Pos) {
> auto FileContents = DraftMgr.getDraft(File);
> assert(FileContents.Draft && "codeComplete is called for non-added
> document");
>
> std::vector<CompletionItem> Result;
> + auto VFS = FSProvider->getFileSystem();
> Units.runOnUnitWithoutReparse(
> - File, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit &Unit) {
> - Result = Unit.codeComplete(*FileContents.Draft, Pos);
> + File, *FileContents.Draft, *CDB, PCHs, VFS, [&](ClangdUnit &Unit) {
> + Result = Unit.codeComplete(*FileContents.Draft, Pos, VFS);
> });
> return Result;
> }
>
> Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
> +++ clang-tools-extra/trunk/clangd/ClangdServer.h Fri May 26 07:26:51 2017
> @@ -50,6 +50,17 @@ public:
> std::vector<DiagWithFixIts> Diagnostics) =
> 0;
> };
>
> +class FileSystemProvider {
> +public:
> + virtual ~FileSystemProvider() = default;
> + virtual IntrusiveRefCntPtr<vfs::FileSystem> getFileSystem() = 0;
> +};
> +
> +class RealFileSystemProvider : public FileSystemProvider {
> +public:
> + IntrusiveRefCntPtr<vfs::FileSystem> getFileSystem() override;
> +};
> +
> class ClangdServer;
>
> /// Handles running WorkerRequests of ClangdServer on a separate threads.
> @@ -94,6 +105,7 @@ class ClangdServer {
> public:
> ClangdServer(std::unique_ptr<GlobalCompilationDatabase> CDB,
> std::unique_ptr<DiagnosticsConsumer> DiagConsumer,
> + std::unique_ptr<FileSystemProvider> FSProvider,
> bool RunSynchronously);
>
> /// Add a \p File to the list of tracked C++ files or update the contents if
> @@ -104,6 +116,8 @@ public:
> /// Remove \p File from list of tracked files, schedule a request to free
> /// resources associated with it.
> void removeDocument(PathRef File);
> + /// Force \p File to be reparsed using the latest contents.
> + void forceReparse(PathRef File);
>
> /// Run code completion for \p File at \p Pos.
> std::vector<CompletionItem> codeComplete(PathRef File, Position Pos);
> @@ -129,6 +143,7 @@ public:
> private:
> std::unique_ptr<GlobalCompilationDatabase> CDB;
> std::unique_ptr<DiagnosticsConsumer> DiagConsumer;
> + std::unique_ptr<FileSystemProvider> FSProvider;
> DraftStore DraftMgr;
> ClangdUnitStore Units;
> std::shared_ptr<PCHContainerOperations> PCHs;
>
> Modified: clang-tools-extra/trunk/clangd/ClangdUnit.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.cpp?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/clangd/ClangdUnit.cpp (original)
> +++ clang-tools-extra/trunk/clangd/ClangdUnit.cpp Fri May 26 07:26:51 2017
> @@ -11,6 +11,7 @@
> #include "clang/Frontend/ASTUnit.h"
> #include "clang/Frontend/CompilerInstance.h"
> #include "clang/Frontend/CompilerInvocation.h"
> +#include "clang/Frontend/Utils.h"
> #include "clang/Tooling/CompilationDatabase.h"
>
> using namespace clang::clangd;
> @@ -18,7 +19,8 @@ using namespace clang;
>
> ClangdUnit::ClangdUnit(PathRef FileName, StringRef Contents,
> std::shared_ptr<PCHContainerOperations> PCHs,
> - std::vector<tooling::CompileCommand> Commands)
> + std::vector<tooling::CompileCommand> Commands,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS)
> : FileName(FileName), PCHs(PCHs) {
> assert(!Commands.empty() && "No compile commands provided");
>
> @@ -48,10 +50,16 @@ ClangdUnit::ClangdUnit(PathRef FileName,
> /*PrecompilePreambleAfterNParses=*/1, /*TUKind=*/TU_Prefix,
> /*CacheCodeCompletionResults=*/true,
> /*IncludeBriefCommentsInCodeCompletion=*/true,
> - /*AllowPCHWithCompilerErrors=*/true));
> + /*AllowPCHWithCompilerErrors=*/true,
> + /*SkipFunctionBodies=*/false,
> + /*UserFilesAreVolatile=*/false, /*ForSerialization=*/false,
> + /*ModuleFormat=*/llvm::None,
> + /*ErrAST=*/nullptr, VFS));
> + assert(Unit && "Unit wasn't created");
> }
>
> -void ClangdUnit::reparse(StringRef Contents) {
> +void ClangdUnit::reparse(StringRef Contents,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS) {
> // Do a reparse if this wasn't the first parse.
> // FIXME: This might have the wrong working directory if it changed in the
> // meantime.
> @@ -59,7 +67,7 @@ void ClangdUnit::reparse(StringRef Conte
> FileName,
> llvm::MemoryBuffer::getMemBufferCopy(Contents, FileName).release());
>
> - Unit->Reparse(PCHs, RemappedSource);
> + Unit->Reparse(PCHs, RemappedSource, VFS);
> }
>
> namespace {
> @@ -146,8 +154,9 @@ public:
> };
> } // namespace
>
> -std::vector<CompletionItem> ClangdUnit::codeComplete(StringRef Contents,
> - Position Pos) {
> +std::vector<CompletionItem>
> +ClangdUnit::codeComplete(StringRef Contents, Position Pos,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS) {
> CodeCompleteOptions CCO;
> CCO.IncludeBriefComments = 1;
> // This is where code completion stores dirty buffers. Need to free after
> @@ -163,8 +172,10 @@ std::vector<CompletionItem> ClangdUnit::
> FileName,
> llvm::MemoryBuffer::getMemBufferCopy(Contents, FileName).release());
>
> + IntrusiveRefCntPtr<FileManager> FileMgr(
> + new FileManager(Unit->getFileSystemOpts(), VFS));
> IntrusiveRefCntPtr<SourceManager> SourceMgr(
> - new SourceManager(*DiagEngine, Unit->getFileManager()));
> + new SourceManager(*DiagEngine, *FileMgr));
> // CodeComplete seems to require fresh LangOptions.
> LangOptions LangOpts = Unit->getLangOpts();
> // The language server protocol uses zero-based line and column numbers.
> @@ -172,8 +183,8 @@ std::vector<CompletionItem> ClangdUnit::
> Unit->CodeComplete(FileName, Pos.line + 1, Pos.character + 1,
> RemappedSource,
> CCO.IncludeMacros, CCO.IncludeCodePatterns,
> CCO.IncludeBriefComments, Collector, PCHs, *DiagEngine,
> - LangOpts, *SourceMgr, Unit->getFileManager(),
> - StoredDiagnostics, OwnedBuffers);
> + LangOpts, *SourceMgr, *FileMgr, StoredDiagnostics,
> + OwnedBuffers);
> for (const llvm::MemoryBuffer *Buffer : OwnedBuffers)
> delete Buffer;
> return Items;
>
> Modified: clang-tools-extra/trunk/clangd/ClangdUnit.h
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.h?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/clangd/ClangdUnit.h (original)
> +++ clang-tools-extra/trunk/clangd/ClangdUnit.h Fri May 26 07:26:51 2017
> @@ -10,8 +10,8 @@
> #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDUNIT_H
> #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDUNIT_H
>
> -#include "Protocol.h"
> #include "Path.h"
> +#include "Protocol.h"
> #include "clang/Frontend/ASTUnit.h"
> #include "clang/Tooling/Core/Replacement.h"
> #include <memory>
> @@ -24,6 +24,10 @@ namespace clang {
> class ASTUnit;
> class PCHContainerOperations;
>
> +namespace vfs {
> +class FileSystem;
> +}
> +
> namespace tooling {
> struct CompileCommand;
> }
> @@ -42,16 +46,19 @@ class ClangdUnit {
> public:
> ClangdUnit(PathRef FileName, StringRef Contents,
> std::shared_ptr<PCHContainerOperations> PCHs,
> - std::vector<tooling::CompileCommand> Commands);
> + std::vector<tooling::CompileCommand> Commands,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS);
>
> /// Reparse with new contents.
> - void reparse(StringRef Contents);
> + void reparse(StringRef Contents, IntrusiveRefCntPtr<vfs::FileSystem> VFS);
>
> /// Get code completions at a specified \p Line and \p Column in \p File.
> ///
> /// This function is thread-safe and returns completion items that own the
> /// data they contain.
> - std::vector<CompletionItem> codeComplete(StringRef Contents, Position Pos);
> + std::vector<CompletionItem>
> + codeComplete(StringRef Contents, Position Pos,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS);
> /// Returns diagnostics and corresponding FixIts for each diagnostic that
> are
> /// located in the current file.
> std::vector<DiagWithFixIts> getLocalDiagnostics() const;
>
> Modified: clang-tools-extra/trunk/clangd/ClangdUnitStore.h
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnitStore.h?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/clangd/ClangdUnitStore.h (original)
> +++ clang-tools-extra/trunk/clangd/ClangdUnitStore.h Fri May 26 07:26:51 2017
> @@ -32,9 +32,10 @@ public:
> template <class Func>
> void runOnUnit(PathRef File, StringRef FileContents,
> GlobalCompilationDatabase &CDB,
> - std::shared_ptr<PCHContainerOperations> PCHs, Func Action) {
> + std::shared_ptr<PCHContainerOperations> PCHs,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS, Func Action) {
> runOnUnitImpl(File, FileContents, CDB, PCHs, /*ReparseBeforeAction=*/true,
> - std::forward<Func>(Action));
> + VFS, std::forward<Func>(Action));
> }
>
> /// Run specified \p Action on the ClangdUnit for \p File.
> @@ -45,9 +46,10 @@ public:
> void runOnUnitWithoutReparse(PathRef File, StringRef FileContents,
> GlobalCompilationDatabase &CDB,
> std::shared_ptr<PCHContainerOperations> PCHs,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS,
> Func Action) {
> runOnUnitImpl(File, FileContents, CDB, PCHs,
> /*ReparseBeforeAction=*/false,
> - std::forward<Func>(Action));
> + VFS, std::forward<Func>(Action));
> }
>
> /// Run the specified \p Action on the ClangdUnit for \p File.
> @@ -71,24 +73,23 @@ private:
> void runOnUnitImpl(PathRef File, StringRef FileContents,
> GlobalCompilationDatabase &CDB,
> std::shared_ptr<PCHContainerOperations> PCHs,
> - bool ReparseBeforeAction, Func Action) {
> + bool ReparseBeforeAction,
> + IntrusiveRefCntPtr<vfs::FileSystem> VFS, Func Action) {
> std::lock_guard<std::mutex> Lock(Mutex);
>
> auto Commands = getCompileCommands(CDB, File);
> assert(!Commands.empty() &&
> "getCompileCommands should add default command");
> - // chdir. This is thread hostile.
> - // FIXME(ibiryukov): get rid of this
> - llvm::sys::fs::set_current_path(Commands.front().Directory);
> + VFS->setCurrentWorkingDirectory(Commands.front().Directory);
>
> auto It = OpenedFiles.find(File);
> if (It == OpenedFiles.end()) {
> It = OpenedFiles
> .insert(std::make_pair(
> - File, ClangdUnit(File, FileContents, PCHs, Commands)))
> + File, ClangdUnit(File, FileContents, PCHs, Commands,
> VFS)))
> .first;
> } else if (ReparseBeforeAction) {
> - It->second.reparse(FileContents);
> + It->second.reparse(FileContents, VFS);
> }
> return Action(It->second);
> }
>
> Modified: clang-tools-extra/trunk/unittests/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/CMakeLists.txt?rev=303977&r1=303976&r2=303977&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/CMakeLists.txt (original)
> +++ clang-tools-extra/trunk/unittests/CMakeLists.txt Fri May 26 07:26:51 2017
> @@ -11,4 +11,5 @@ add_subdirectory(clang-move)
> add_subdirectory(clang-query)
> add_subdirectory(clang-tidy)
> add_subdirectory(clang-rename)
> +add_subdirectory(clangd)
> add_subdirectory(include-fixer)
>
> Added: clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt?rev=303977&view=auto
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt (added)
> +++ clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt Fri May 26
> 07:26:51 2017
> @@ -0,0 +1,24 @@
> +set(LLVM_LINK_COMPONENTS
> + support
> + )
> +
> +get_filename_component(CLANGD_SOURCE_DIR
> + ${CMAKE_CURRENT_SOURCE_DIR}/../../clangd REALPATH)
> +include_directories(
> + ${CLANGD_SOURCE_DIR}
> + )
> +
> +add_extra_unittest(ClangdTests
> + ClangdTests.cpp
> + )
> +
> +target_link_libraries(ClangdTests
> + clangBasic
> + clangDaemon
> + clangFormat
> + clangFrontend
> + clangSema
> + clangTooling
> + clangToolingCore
> + LLVMSupport
> + )
>
> Added: clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp?rev=303977&view=auto
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp (added)
> +++ clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp Fri May 26
> 07:26:51 2017
> @@ -0,0 +1,364 @@
> +//===-- ClangdTests.cpp - Clangd unit tests ---------------------*- C++
> -*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "ClangdServer.h"
> +#include "clang/Basic/VirtualFileSystem.h"
> +#include "llvm/ADT/SmallVector.h"
> +#include "llvm/ADT/StringMap.h"
> +#include "llvm/Config/config.h"
> +#include "llvm/Support/Errc.h"
> +#include "llvm/Support/Path.h"
> +#include "llvm/Support/Regex.h"
> +#include "gtest/gtest.h"
> +#include <algorithm>
> +#include <iostream>
> +#include <string>
> +#include <vector>
> +
> +namespace clang {
> +namespace vfs {
> +
> +/// An implementation of vfs::FileSystem that only allows access to
> +/// files and folders inside a set of whitelisted directories.
> +///
> +/// FIXME(ibiryukov): should it also emulate access to parents of whitelisted
> +/// directories with only whitelisted contents?
> +class FilteredFileSystem : public vfs::FileSystem {
> +public:
> + /// The paths inside \p WhitelistedDirs should be absolute
> + FilteredFileSystem(std::vector<std::string> WhitelistedDirs,
> + IntrusiveRefCntPtr<vfs::FileSystem> InnerFS)
> + : WhitelistedDirs(std::move(WhitelistedDirs)), InnerFS(InnerFS) {
> + assert(std::all_of(WhitelistedDirs.begin(), WhitelistedDirs.end(),
> + [](const std::string &Path) -> bool {
> + return llvm::sys::path::is_absolute(Path);
> + }) &&
> + "Not all WhitelistedDirs are absolute");
> + }
> +
> + virtual llvm::ErrorOr<Status> status(const Twine &Path) {
> + if (!isInsideWhitelistedDir(Path))
> + return llvm::errc::no_such_file_or_directory;
> + return InnerFS->status(Path);
> + }
> +
> + virtual llvm::ErrorOr<std::unique_ptr<File>>
> + openFileForRead(const Twine &Path) {
> + if (!isInsideWhitelistedDir(Path))
> + return llvm::errc::no_such_file_or_directory;
> + return InnerFS->openFileForRead(Path);
> + }
> +
> + llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>>
> + getBufferForFile(const Twine &Name, int64_t FileSize = -1,
> + bool RequiresNullTerminator = true,
> + bool IsVolatile = false) {
> + if (!isInsideWhitelistedDir(Name))
> + return llvm::errc::no_such_file_or_directory;
> + return InnerFS->getBufferForFile(Name, FileSize, RequiresNullTerminator,
> + IsVolatile);
> + }
> +
> + virtual directory_iterator dir_begin(const Twine &Dir, std::error_code
> &EC) {
> + if (!isInsideWhitelistedDir(Dir)) {
> + EC = llvm::errc::no_such_file_or_directory;
> + return directory_iterator();
> + }
> + return InnerFS->dir_begin(Dir, EC);
> + }
> +
> + virtual std::error_code setCurrentWorkingDirectory(const Twine &Path) {
> + return InnerFS->setCurrentWorkingDirectory(Path);
> + }
> +
> + virtual llvm::ErrorOr<std::string> getCurrentWorkingDirectory() const {
> + return InnerFS->getCurrentWorkingDirectory();
> + }
> +
> + bool exists(const Twine &Path) {
> + if (!isInsideWhitelistedDir(Path))
> + return false;
> + return InnerFS->exists(Path);
> + }
> +
> + std::error_code makeAbsolute(SmallVectorImpl<char> &Path) const {
> + return InnerFS->makeAbsolute(Path);
> + }
> +
> +private:
> + bool isInsideWhitelistedDir(const Twine &InputPath) const {
> + SmallString<128> Path;
> + InputPath.toVector(Path);
> +
> + if (makeAbsolute(Path))
> + return false;
> +
> + for (const auto &Dir : WhitelistedDirs) {
> + if (Path.startswith(Dir))
> + return true;
> + }
> + return false;
> + }
> +
> + std::vector<std::string> WhitelistedDirs;
> + IntrusiveRefCntPtr<vfs::FileSystem> InnerFS;
> +};
> +
> +/// Create a vfs::FileSystem that has access only to temporary directories
> +/// (obtained by calling system_temp_directory).
> +IntrusiveRefCntPtr<vfs::FileSystem> getTempOnlyFS() {
> + llvm::SmallString<128> TmpDir1;
> + llvm::sys::path::system_temp_directory(/*erasedOnReboot=*/false, TmpDir1);
> + llvm::SmallString<128> TmpDir2;
> + llvm::sys::path::system_temp_directory(/*erasedOnReboot=*/true, TmpDir2);
> +
> + std::vector<std::string> TmpDirs;
> + TmpDirs.push_back(TmpDir1.str());
> + if (TmpDir2 != TmpDir2)
> + TmpDirs.push_back(TmpDir2.str());
> + return new vfs::FilteredFileSystem(std::move(TmpDirs),
> + vfs::getRealFileSystem());
> +}
> +} // namespace vfs
> +
> +namespace clangd {
> +namespace {
> +
> +class ErrorCheckingDiagConsumer : public DiagnosticsConsumer {
> +public:
> + void onDiagnosticsReady(PathRef File,
> + std::vector<DiagWithFixIts> Diagnostics) override {
> + bool HadError = false;
> + for (const auto &DiagAndFixIts : Diagnostics) {
> + // FIXME: severities returned by clangd should have a descriptive
> + // diagnostic severity enum
> + const int ErrorSeverity = 1;
> + HadError = DiagAndFixIts.Diag.severity == ErrorSeverity;
> + }
> +
> + std::lock_guard<std::mutex> Lock(Mutex);
> + HadErrorInLastDiags = HadError;
> + }
> +
> + bool hadErrorInLastDiags() {
> + std::lock_guard<std::mutex> Lock(Mutex);
> + return HadErrorInLastDiags;
> + }
> +
> +private:
> + std::mutex Mutex;
> + bool HadErrorInLastDiags = false;
> +};
> +
> +class MockCompilationDatabase : public GlobalCompilationDatabase {
> +public:
> + std::vector<tooling::CompileCommand>
> + getCompileCommands(PathRef File) override {
> + return {};
> + }
> +};
> +
> +class MockFSProvider : public FileSystemProvider {
> +public:
> + IntrusiveRefCntPtr<vfs::FileSystem> getFileSystem() override {
> + IntrusiveRefCntPtr<vfs::InMemoryFileSystem> MemFS(
> + new vfs::InMemoryFileSystem);
> + for (auto &FileAndContents : Files)
> + MemFS->addFile(FileAndContents.first(), time_t(),
> + llvm::MemoryBuffer::getMemBuffer(FileAndContents.second,
> +
> FileAndContents.first()));
> +
> + auto OverlayFS = IntrusiveRefCntPtr<vfs::OverlayFileSystem>(
> + new vfs::OverlayFileSystem(vfs::getTempOnlyFS()));
> + OverlayFS->pushOverlay(std::move(MemFS));
> + return OverlayFS;
> + }
> +
> + llvm::StringMap<std::string> Files;
> +};
> +
> +/// Replaces all patterns of the form 0x123abc with spaces
> +void replacePtrsInDump(std::string &Dump) {
> + llvm::Regex RE("0x[0-9a-fA-F]+");
> + llvm::SmallVector<StringRef, 1> Matches;
> + while (RE.match(Dump, &Matches)) {
> + assert(Matches.size() == 1 && "Exactly one match expected");
> + auto MatchPos = Matches[0].data() - Dump.data();
> + std::fill(Dump.begin() + MatchPos,
> + Dump.begin() + MatchPos + Matches[0].size(), ' ');
> + }
> +}
> +
> +std::string dumpASTWithoutMemoryLocs(ClangdServer &Server, PathRef File) {
> + auto Dump = Server.dumpAST(File);
> + replacePtrsInDump(Dump);
> + return Dump;
> +}
> +
> +template <class T>
> +std::unique_ptr<T> getAndMove(std::unique_ptr<T> Ptr, T *&Output) {
> + Output = Ptr.get();
> + return Ptr;
> +}
> +} // namespace
> +
> +class ClangdVFSTest : public ::testing::Test {
> +protected:
> + SmallString<16> getVirtualTestRoot() {
> +#ifdef LLVM_ON_WIN32
> + return SmallString<16>("C:\\clangd-test");
> +#else
> + return SmallString<16>("/clangd-test");
> +#endif
> + }
> +
> + llvm::SmallString<32> getVirtualTestFilePath(PathRef File) {
> + assert(llvm::sys::path::is_relative(File) && "FileName should be
> relative");
> +
> + llvm::SmallString<32> Path;
> + llvm::sys::path::append(Path, getVirtualTestRoot(), File);
> + return Path;
> + }
> +
> + std::string parseSourceAndDumpAST(
> + PathRef SourceFileRelPath, StringRef SourceContents,
> + std::vector<std::pair<PathRef, StringRef>> ExtraFiles = {},
> + bool ExpectErrors = false) {
> + MockFSProvider *FS;
> + ErrorCheckingDiagConsumer *DiagConsumer;
> + ClangdServer Server(
> + llvm::make_unique<MockCompilationDatabase>(),
> + getAndMove(llvm::make_unique<ErrorCheckingDiagConsumer>(),
> + DiagConsumer),
> + getAndMove(llvm::make_unique<MockFSProvider>(), FS),
> + /*RunSynchronously=*/false);
> + for (const auto &FileWithContents : ExtraFiles)
> + FS->Files[getVirtualTestFilePath(FileWithContents.first)] =
> + FileWithContents.second;
> +
> + auto SourceFilename = getVirtualTestFilePath(SourceFileRelPath);
> + Server.addDocument(SourceFilename, SourceContents);
> + auto Result = dumpASTWithoutMemoryLocs(Server, SourceFilename);
> + EXPECT_EQ(ExpectErrors, DiagConsumer->hadErrorInLastDiags());
> + return Result;
> + }
> +};
> +
> +TEST_F(ClangdVFSTest, Parse) {
> + // FIXME: figure out a stable format for AST dumps, so that we can check
> the
> + // output of the dump itself is equal to the expected one, not just that
> it's
> + // different.
> + auto Empty = parseSourceAndDumpAST("foo.cpp", "", {});
> + auto OneDecl = parseSourceAndDumpAST("foo.cpp", "int a;", {});
> + auto SomeDecls = parseSourceAndDumpAST("foo.cpp", "int a; int b; int c;",
> {});
> + EXPECT_NE(Empty, OneDecl);
> + EXPECT_NE(Empty, SomeDecls);
> + EXPECT_NE(SomeDecls, OneDecl);
> +
> + auto Empty2 = parseSourceAndDumpAST("foo.cpp", "");
> + auto OneDecl2 = parseSourceAndDumpAST("foo.cpp", "int a;");
> + auto SomeDecls2 = parseSourceAndDumpAST("foo.cpp", "int a; int b; int c;");
> + EXPECT_EQ(Empty, Empty2);
> + EXPECT_EQ(OneDecl, OneDecl2);
> + EXPECT_EQ(SomeDecls, SomeDecls2);
> +}
> +
> +TEST_F(ClangdVFSTest, ParseWithHeader) {
> + parseSourceAndDumpAST("foo.cpp", "#include \"foo.h\"", {},
> + /*ExpectErrors=*/true);
> + parseSourceAndDumpAST("foo.cpp", "#include \"foo.h\"", {{"foo.h", ""}},
> + /*ExpectErrors=*/false);
> +
> + const auto SourceContents = R"cpp(
> +#include "foo.h"
> +int b = a;
> +)cpp";
> + parseSourceAndDumpAST("foo.cpp", SourceContents, {{"foo.h", ""}},
> + /*ExpectErrors=*/true);
> + parseSourceAndDumpAST("foo.cpp", SourceContents, {{"foo.h", "int a;"}},
> + /*ExpectErrors=*/false);
> +}
> +
> +TEST_F(ClangdVFSTest, Reparse) {
> + MockFSProvider *FS;
> + ErrorCheckingDiagConsumer *DiagConsumer;
> + ClangdServer Server(
> + llvm::make_unique<MockCompilationDatabase>(),
> + getAndMove(llvm::make_unique<ErrorCheckingDiagConsumer>(),
> DiagConsumer),
> + getAndMove(llvm::make_unique<MockFSProvider>(), FS),
> + /*RunSynchronously=*/false);
> +
> + const auto SourceContents = R"cpp(
> +#include "foo.h"
> +int b = a;
> +)cpp";
> +
> + auto FooCpp = getVirtualTestFilePath("foo.cpp");
> + auto FooH = getVirtualTestFilePath("foo.h");
> +
> + FS->Files[FooH] = "int a;";
> + FS->Files[FooCpp] = SourceContents;
> +
> + Server.addDocument(FooCpp, SourceContents);
> + auto DumpParse1 = dumpASTWithoutMemoryLocs(Server, FooCpp);
> + EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());
> +
> + Server.addDocument(FooCpp, "");
> + auto DumpParseEmpty = dumpASTWithoutMemoryLocs(Server, FooCpp);
> + EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());
> +
> + Server.addDocument(FooCpp, SourceContents);
> + auto DumpParse2 = dumpASTWithoutMemoryLocs(Server, FooCpp);
> + EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());
> +
> + EXPECT_EQ(DumpParse1, DumpParse2);
> + EXPECT_NE(DumpParse1, DumpParseEmpty);
> +}
> +
> +TEST_F(ClangdVFSTest, ReparseOnHeaderChange) {
> + MockFSProvider *FS;
> + ErrorCheckingDiagConsumer *DiagConsumer;
> +
> + ClangdServer Server(
> + llvm::make_unique<MockCompilationDatabase>(),
> + getAndMove(llvm::make_unique<ErrorCheckingDiagConsumer>(),
> DiagConsumer),
> + getAndMove(llvm::make_unique<MockFSProvider>(), FS),
> + /*RunSynchronously=*/false);
> +
> + const auto SourceContents = R"cpp(
> +#include "foo.h"
> +int b = a;
> +)cpp";
> +
> + auto FooCpp = getVirtualTestFilePath("foo.cpp");
> + auto FooH = getVirtualTestFilePath("foo.h");
> +
> + FS->Files[FooH] = "int a;";
> + FS->Files[FooCpp] = SourceContents;
> +
> + Server.addDocument(FooCpp, SourceContents);
> + auto DumpParse1 = dumpASTWithoutMemoryLocs(Server, FooCpp);
> + EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());
> +
> + FS->Files[FooH] = "";
> + Server.forceReparse(FooCpp);
> + auto DumpParseDifferent = dumpASTWithoutMemoryLocs(Server, FooCpp);
> + EXPECT_TRUE(DiagConsumer->hadErrorInLastDiags());
> +
> + FS->Files[FooH] = "int a;";
> + Server.forceReparse(FooCpp);
> + auto DumpParse2 = dumpASTWithoutMemoryLocs(Server, FooCpp);
> + EXPECT_FALSE(DiagConsumer->hadErrorInLastDiags());
> +
> + EXPECT_EQ(DumpParse1, DumpParse2);
> + EXPECT_NE(DumpParse1, DumpParseDifferent);
> +}
> +
> +} // namespace clangd
> +} // namespace clang
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
