Thanks! There was actually some old debugging code that got left over in a test. Sorry about that! Fixed in r343845.
On Fri, Oct 5, 2018 at 11:34 AM <douglas.y...@sony.com> wrote: > Hi Sam, > > Our internal build bot hit an assertion failure in the changes you made in > this commit that I was able to reproduce by building your change on my > machine on Windows 7 using Visual Studio 2015. None of the public build > bots seem to have hit this issue, though I am not sure why at the moment. > > Here is the assertion failure that is hit while running ClangdTests.exe: > > [----------] 2 tests from SerializationTest > [ RUN ] SerializationTest.YAMLConversions > [ OK ] SerializationTest.YAMLConversions (0 ms) > [ RUN ] SerializationTest.BinaryConversions > Assertion failed: OutBufCur == OutBufStart && "raw_ostream destructor > called with non-empty buffer!", file > C:\src\upstream\llvm2\lib\Support\raw_ostream.cpp, line 73 > 0x000000013FA74405 (0x0000000000000016 0x0000000000000200 > 0x000006D100230006 0x000007FEEA9BAA51), HandleAbort() + 0x5 bytes(s), > c:\src\upstream\llvm2\lib\support\windows\signals.inc, line 409 > 0x000007FEEAA1DC17 (0x0000000000000001 0x0000000100000000 > 0x0000000000000000 0x00000000009BF460), raise() + 0x1E7 bytes(s) > 0x000007FEEAA1EAA1 (0x000007FE00000003 0x000007FE00000003 > 0x00000001413AF810 0x00000001413AF790), abort() + 0x31 bytes(s) > 0x000007FEEAA20751 (0x0000000000000049 0x00000001413AF810 > 0x0000000000000122 0x000007FEEA9C05D6), _get_wpgmptr() + 0x1BE1 bytes(s) > 0x000007FEEAA20A5F (0x00000000009BF890 0x00000000009BF680 > 0x0000000000B05AB0 0x000000014126F8C5), _wassert() + 0x3F bytes(s) > 0x000000013FA40677 (0x0000000000000000 0x0000000000A80000 > 0x00000000009BF890 0x000000013FA41A2E), llvm::raw_ostream::~raw_ostream() + > 0x37 bytes(s), c:\src\upstream\llvm2\lib\support\raw_ostream.cpp, line 75 > 0x000000013FA4057C (0x0000000004BA1FF0 0x00000000009BF680 > 0x0000000000B05AB0 0x0000000000B05AB0), > llvm::raw_fd_ostream::~raw_fd_ostream() + 0x11C bytes(s), > c:\src\upstream\llvm2\lib\support\raw_ostream.cpp, line 617 > 0x000000013F96A4CC (0x0000000004BA1FF0 0x0000000000B05AB0 > 0x0000000000B05AB0 0x0000000004BA1FF0), clang::clangd::`anonymous > namespace'::SerializationTest_BinaryConversions_Test::TestBody() + 0x34C > bytes(s), > c:\src\upstream\llvm2\tools\clang\tools\extra\unittests\clangd\serializationtests.cpp, > line 166 > 0x000000013FAE201C (0x0000000004BA1FF0 0x0000000000B05AB0 > 0x00000001413C4B68 0x000007FEEAA954B8), > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void>() > + 0xC bytes(s), > c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, line 2387 + > 0x2 byte(s) > 0x000000013FB09E5B (0x0000000004BA1FF0 0x0000000000B05AB0 > 0x00000001413C4D30 0x00000000009BF9E8), testing::Test::Run() + 0x7B > bytes(s), c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, > line 2481 > 0x000000013FB0A0DB (0x0000016641B94CAF 0x0000000000B05AB0 > 0x0000000000B1DE30 0x0000000000AFF560), testing::TestInfo::Run() + 0xAB > bytes(s), c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, > line 2660 > 0x000000013FB09F72 (0x0000000000000023 0x0000000000000000 > 0x0000000000B05AB0 0x0000000000AFF560), testing::TestCase::Run() + 0xB2 > bytes(s), c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, > line 2774 + 0x12 byte(s) > 0x000000013FB0A529 (0x0000000000AFF410 0x0000000000000000 > 0x0000000000000001 0x000007FE00000001), > testing::internal::UnitTestImpl::RunAllTests() + 0x229 bytes(s), > c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, line 4649 + > 0x39 byte(s) > 0x000000013FAE213C (0x0000000000AFF410 0x0000000000000000 > 0x00000001413C54A8 0x0000000000000000), > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>() > + 0xC bytes(s), > c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, line 2387 + > 0x2 byte(s) > 0x000000013FB0A27C (0x0000000000008002 0x0000000000000000 > 0x0000000000000000 0x0000000000000000), testing::UnitTest::Run() + 0xEC > bytes(s), c:\src\upstream\llvm2\utils\unittest\googletest\src\gtest.cc, > line 4257 + 0x17 byte(s) > 0x0000000141299397 (0x0000000100000001 0x0000000000000000 > 0x0000000000000000 0x01D45C4625BC735E), main() + 0xB7 bytes(s), > c:\src\upstream\llvm2\utils\unittest\unittestmain\testmain.cpp, line 52 > 0x00000001412707C1 (0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000), __scrt_common_main_seh() + 0x11D > bytes(s), f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl, line 253 > + 0x22 byte(s) > 0x0000000076C159CD (0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000), BaseThreadInitThunk() + 0xD bytes(s) > 0x0000000076E7385D (0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000), RtlUserThreadStart() + 0x1D bytes(s) > > Could you take a look into this? > > Douglas Yung > > > -----Original Message----- > > From: cfe-commits [mailto:cfe-commits-boun...@lists.llvm.org] On Behalf > > Of Sam McCall via cfe-commits > > Sent: Thursday, October 04, 2018 7:10 > > To: cfe-commits@lists.llvm.org > > Subject: [clang-tools-extra] r343778 - [clangd] clangd-indexer gathers > > refs and stores them in index files. > > > > Author: sammccall > > Date: Thu Oct 4 07:09:55 2018 > > New Revision: 343778 > > > > URL: http://llvm.org/viewvc/llvm-project?rev=343778&view=rev > > Log: > > [clangd] clangd-indexer gathers refs and stores them in index files. > > > > Reviewers: ioeric > > > > Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe- > > commits > > > > Differential Revision: https://reviews.llvm.org/D52531 > > > > Modified: > > clang-tools-extra/trunk/clangd/index/IndexAction.cpp > > clang-tools-extra/trunk/clangd/index/IndexAction.h > > clang-tools-extra/trunk/clangd/index/Serialization.cpp > > clang-tools-extra/trunk/clangd/index/Serialization.h > > clang-tools-extra/trunk/clangd/index/YAMLSerialization.cpp > > clang-tools-extra/trunk/clangd/indexer/IndexerMain.cpp > > clang-tools-extra/trunk/unittests/clangd/SerializationTests.cpp > > > > Modified: clang-tools-extra/trunk/clangd/index/IndexAction.cpp > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/clangd/index/IndexAction.cpp?rev=343778&r1=343777&r2=343778 > > &view=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/clangd/index/IndexAction.cpp (original) > > +++ clang-tools-extra/trunk/clangd/index/IndexAction.cpp Thu Oct 4 > > 07:09:55 2018 > > @@ -13,10 +13,11 @@ public: > > IndexAction(std::shared_ptr<SymbolCollector> C, > > std::unique_ptr<CanonicalIncludes> Includes, > > const index::IndexingOptions &Opts, > > - std::function<void(SymbolSlab)> &SymbolsCallback) > > + std::function<void(SymbolSlab)> SymbolsCallback, > > + std::function<void(RefSlab)> RefsCallback) > > : WrapperFrontendAction(index::createIndexingAction(C, Opts, > > nullptr)), > > - SymbolsCallback(SymbolsCallback), Collector(C), > > - Includes(std::move(Includes)), > > + SymbolsCallback(SymbolsCallback), RefsCallback(RefsCallback), > > + Collector(C), Includes(std::move(Includes)), > > PragmaHandler(collectIWYUHeaderMaps(this->Includes.get())) {} > > > > std::unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &CI, > > @@ -41,10 +42,13 @@ public: > > return; > > } > > SymbolsCallback(Collector->takeSymbols()); > > + if (RefsCallback != nullptr) > > + RefsCallback(Collector->takeRefs()); > > } > > > > private: > > std::function<void(SymbolSlab)> SymbolsCallback; > > + std::function<void(RefSlab)> RefsCallback; > > std::shared_ptr<SymbolCollector> Collector; > > std::unique_ptr<CanonicalIncludes> Includes; > > std::unique_ptr<CommentHandler> PragmaHandler; > > @@ -54,20 +58,23 @@ private: > > > > std::unique_ptr<FrontendAction> > > createStaticIndexingAction(SymbolCollector::Options Opts, > > - std::function<void(SymbolSlab)> > > SymbolsCallback) { > > + std::function<void(SymbolSlab)> > > SymbolsCallback, > > + std::function<void(RefSlab)> RefsCallback) > > { > > index::IndexingOptions IndexOpts; > > IndexOpts.SystemSymbolFilter = > > index::IndexingOptions::SystemSymbolFilterKind::All; > > Opts.CollectIncludePath = true; > > Opts.CountReferences = true; > > Opts.Origin = SymbolOrigin::Static; > > + if (RefsCallback != nullptr) > > + Opts.RefFilter = RefKind::All; > > auto Includes = llvm::make_unique<CanonicalIncludes>(); > > addSystemHeadersMapping(Includes.get()); > > Opts.Includes = Includes.get(); > > return llvm::make_unique<IndexAction>( > > std::make_shared<SymbolCollector>(std::move(Opts)), > > std::move(Includes), > > - IndexOpts, SymbolsCallback); > > -} > > + IndexOpts, SymbolsCallback, RefsCallback); > > +}; > > > > } // namespace clangd > > } // namespace clang > > > > Modified: clang-tools-extra/trunk/clangd/index/IndexAction.h > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/clangd/index/IndexAction.h?rev=343778&r1=343777&r2=343778&v > > iew=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/clangd/index/IndexAction.h (original) > > +++ clang-tools-extra/trunk/clangd/index/IndexAction.h Thu Oct 4 > > 07:09:55 2018 > > @@ -21,10 +21,13 @@ namespace clangd { > > // Only a subset of SymbolCollector::Options are respected: > > // - include paths are always collected, and canonicalized > > appropriately > > // - references are always counted > > +// - main-file refs are collected (if RefsCallback is non-null) > > // - the symbol origin is always Static > > +// FIXME: refs from headers should also be collected. > > std::unique_ptr<FrontendAction> > > createStaticIndexingAction(SymbolCollector::Options Opts, > > - std::function<void(SymbolSlab)> > > SymbolsCallback); > > + std::function<void(SymbolSlab)> > > SymbolsCallback, > > + std::function<void(RefSlab)> RefsCallback); > > > > } // namespace clangd > > } // namespace clang > > > > Modified: clang-tools-extra/trunk/clangd/index/Serialization.cpp > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/clangd/index/Serialization.cpp?rev=343778&r1=343777&r2=3437 > > 78&view=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/clangd/index/Serialization.cpp (original) > > +++ clang-tools-extra/trunk/clangd/index/Serialization.cpp Thu Oct 4 > > 07:09:55 2018 > > @@ -298,17 +298,47 @@ Symbol readSymbol(Reader &Data, ArrayRef > > return Sym; > > } > > > > +// REFS ENCODING > > +// A refs section has data grouped by Symbol. Each symbol has: > > +// - SymbolID: 20 bytes > > +// - NumRefs: varint > > +// - Ref[NumRefs] > > +// Fields of Ref are encoded in turn, see implementation. > > + > > +void writeRefs(const SymbolID &ID, ArrayRef<Ref> Refs, > > + const StringTableOut &Strings, raw_ostream &OS) { > > + OS << ID.raw(); > > + writeVar(Refs.size(), OS); > > + for (const auto &Ref : Refs) { > > + OS.write(static_cast<unsigned char>(Ref.Kind)); > > + writeLocation(Ref.Location, Strings, OS); > > + } > > +} > > + > > +std::pair<SymbolID, std::vector<Ref>> readRefs(Reader &Data, > > + ArrayRef<StringRef> > > Strings) { > > + std::pair<SymbolID, std::vector<Ref>> Result; > > + Result.first = Data.consumeID(); > > + Result.second.resize(Data.consumeVar()); > > + for (auto &Ref : Result.second) { > > + Ref.Kind = static_cast<RefKind>(Data.consume8()); > > + Ref.Location = readLocation(Data, Strings); > > + } > > + return Result; > > +} > > + > > // FILE ENCODING > > // A file is a RIFF chunk with type 'CdIx'. > > // It contains the sections: > > // - meta: version number > > // - stri: string table > > // - symb: symbols > > +// - refs: references to symbols > > > > // The current versioning scheme is simple - non-current versions are > > rejected. > > // If you make a breaking change, bump this version number to > > invalidate stored > > // data. Later we may want to support some backward compatibility. > > -constexpr static uint32_t Version = 4; > > +constexpr static uint32_t Version = 5; > > > > Expected<IndexFileIn> readRIFF(StringRef Data) { > > auto RIFF = riff::readFile(Data); > > @@ -342,6 +372,18 @@ Expected<IndexFileIn> readRIFF(StringRef > > return makeError("malformed or truncated symbol"); > > Result.Symbols = std::move(Symbols).build(); > > } > > + if (Chunks.count("refs")) { > > + Reader RefsReader(Chunks.lookup("refs")); > > + RefSlab::Builder Refs; > > + while (!RefsReader.eof()) { > > + auto RefsBundle = readRefs(RefsReader, Strings->Strings); > > + for (const auto &Ref : RefsBundle.second) // FIXME: bulk insert? > > + Refs.insert(RefsBundle.first, Ref); > > + } > > + if (RefsReader.err()) > > + return makeError("malformed or truncated refs"); > > + Result.Refs = std::move(Refs).build(); > > + } > > return std::move(Result); > > } > > > > @@ -363,6 +405,14 @@ void writeRIFF(const IndexFileOut &Data, > > Symbols.emplace_back(Sym); > > visitStrings(Symbols.back(), [&](StringRef &S) { > > Strings.intern(S); }); > > } > > + std::vector<std::pair<SymbolID, std::vector<Ref>>> Refs; > > + if (Data.Refs) { > > + for (const auto &Sym : *Data.Refs) { > > + Refs.emplace_back(Sym); > > + for (auto &Ref : Refs.back().second) > > + Strings.intern(Ref.Location.FileURI); > > + } > > + } > > > > std::string StringSection; > > { > > @@ -379,6 +429,16 @@ void writeRIFF(const IndexFileOut &Data, > > } > > RIFF.Chunks.push_back({riff::fourCC("symb"), SymbolSection}); > > > > + std::string RefsSection; > > + if (Data.Refs) { > > + { > > + raw_string_ostream RefsOS(RefsSection); > > + for (const auto &Sym : Refs) > > + writeRefs(Sym.first, Sym.second, Strings, RefsOS); > > + } > > + RIFF.Chunks.push_back({riff::fourCC("refs"), RefsSection}); > > + } > > + > > OS << RIFF; > > } > > > > @@ -428,6 +488,8 @@ std::unique_ptr<SymbolIndex> loadIndex(l > > if (auto I = readIndexFile(Buffer->get()->getBuffer())) { > > if (I->Symbols) > > Symbols = std::move(*I->Symbols); > > + if (I->Refs) > > + Refs = std::move(*I->Refs); > > } else { > > llvm::errs() << "Bad Index: " << llvm::toString(I.takeError()) > > << "\n"; > > return nullptr; > > > > Modified: clang-tools-extra/trunk/clangd/index/Serialization.h > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/clangd/index/Serialization.h?rev=343778&r1=343777&r2=343778 > > &view=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/clangd/index/Serialization.h (original) > > +++ clang-tools-extra/trunk/clangd/index/Serialization.h Thu Oct 4 > > 07:09:55 2018 > > @@ -38,26 +38,29 @@ enum class IndexFileFormat { > > // Holds the contents of an index file that was read. > > struct IndexFileIn { > > llvm::Optional<SymbolSlab> Symbols; > > + llvm::Optional<RefSlab> Refs; > > }; > > -// Parse an index file. The input must be a RIFF container chunk. > > +// Parse an index file. The input must be a RIFF or YAML file. > > llvm::Expected<IndexFileIn> readIndexFile(llvm::StringRef); > > > > // Specifies the contents of an index file to be written. > > struct IndexFileOut { > > - const SymbolSlab *Symbols; > > - // TODO: Support serializing symbol occurrences. > > + const SymbolSlab *Symbols = nullptr; > > + const RefSlab *Refs = nullptr; > > // TODO: Support serializing Dex posting lists. > > IndexFileFormat Format = IndexFileFormat::RIFF; > > > > IndexFileOut() = default; > > IndexFileOut(const IndexFileIn &I) > > - : Symbols(I.Symbols ? I.Symbols.getPointer() : nullptr) {} > > + : Symbols(I.Symbols ? I.Symbols.getPointer() : nullptr), > > + Refs(I.Refs ? I.Refs.getPointer() : nullptr) {} > > }; > > // Serializes an index file. > > llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const > > IndexFileOut &O); > > > > // Convert a single symbol to YAML, a nice debug representation. > > std::string toYAML(const Symbol &); > > +std::string toYAML(const std::pair<SymbolID, ArrayRef<Ref>> &); > > > > // Build an in-memory static index from an index file. > > // The size should be relatively small, so data can be managed in > > memory. > > > > Modified: clang-tools-extra/trunk/clangd/index/YAMLSerialization.cpp > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/clangd/index/YAMLSerialization.cpp?rev=343778&r1=343777&r2= > > 343778&view=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/clangd/index/YAMLSerialization.cpp > > (original) > > +++ clang-tools-extra/trunk/clangd/index/YAMLSerialization.cpp Thu Oct > > 4 07:09:55 2018 > > @@ -6,6 +6,12 @@ > > // License. See LICENSE.TXT for details. > > // > > //===----------------------------------------------------------------- > > -----===// > > +// > > +// A YAML index file is a sequence of tagged entries. > > +// Each entry either encodes a Symbol or the list of references to a > > symbol > > +// (a "ref bundle"). > > +// > > +//===----------------------------------------------------------------- > > -----===// > > > > #include "Index.h" > > #include "Serialization.h" > > @@ -20,10 +26,22 @@ > > #include <cstdint> > > > > > > LLVM_YAML_IS_SEQUENCE_VECTOR(clang::clangd::Symbol::IncludeHeaderWithRe > > ferences) > > +LLVM_YAML_IS_SEQUENCE_VECTOR(clang::clangd::Ref) > > > > +namespace { > > +using RefBundle = > > + std::pair<clang::clangd::SymbolID, > > std::vector<clang::clangd::Ref>>; > > +// This is a pale imitation of std::variant<Symbol, RefBundle> > > +struct VariantEntry { > > + llvm::Optional<clang::clangd::Symbol> Symbol; > > + llvm::Optional<RefBundle> Refs; > > +}; > > +} // namespace > > namespace llvm { > > namespace yaml { > > > > +using clang::clangd::Ref; > > +using clang::clangd::RefKind; > > using clang::clangd::Symbol; > > using clang::clangd::SymbolID; > > using clang::clangd::SymbolLocation; > > @@ -179,6 +197,46 @@ template <> struct ScalarEnumerationTrai > > } > > }; > > > > +template <> struct MappingTraits<RefBundle> { > > + static void mapping(IO &IO, RefBundle &Refs) { > > + MappingNormalization<NormalizedSymbolID, SymbolID> NSymbolID(IO, > > + > > Refs.first); > > + IO.mapRequired("ID", NSymbolID->HexString); > > + IO.mapRequired("References", Refs.second); > > + } > > +}; > > + > > +struct NormalizedRefKind { > > + NormalizedRefKind(IO &) {} > > + NormalizedRefKind(IO &, RefKind O) { Kind = static_cast<uint8_t>(O); > > } > > + > > + RefKind denormalize(IO &) { return static_cast<RefKind>(Kind); } > > + > > + uint8_t Kind = 0; > > +}; > > + > > +template <> struct MappingTraits<Ref> { > > + static void mapping(IO &IO, Ref &R) { > > + MappingNormalization<NormalizedRefKind, RefKind> NKind(IO, > > R.Kind); > > + IO.mapRequired("Kind", NKind->Kind); > > + IO.mapRequired("Location", R.Location); > > + } > > +}; > > + > > +template <> struct MappingTraits<VariantEntry> { > > + static void mapping(IO &IO, VariantEntry &Variant) { > > + if (IO.mapTag("!Symbol", Variant.Symbol.hasValue())) { > > + if (!IO.outputting()) > > + Variant.Symbol.emplace(); > > + MappingTraits<Symbol>::mapping(IO, *Variant.Symbol); > > + } else if (IO.mapTag("!Refs", Variant.Refs.hasValue())) { > > + if (!IO.outputting()) > > + Variant.Refs.emplace(); > > + MappingTraits<RefBundle>::mapping(IO, *Variant.Refs); > > + } > > + } > > +}; > > + > > } // namespace yaml > > } // namespace llvm > > > > @@ -187,23 +245,38 @@ namespace clangd { > > > > void writeYAML(const IndexFileOut &O, raw_ostream &OS) { > > llvm::yaml::Output Yout(OS); > > - for (Symbol Sym : *O.Symbols) // copy: Yout<< requires mutability. > > - Yout << Sym; > > + for (const auto &Sym : *O.Symbols) { > > + VariantEntry Entry; > > + Entry.Symbol = Sym; > > + Yout << Entry; > > + } > > + if (O.Refs) > > + for (auto &Sym : *O.Refs) { > > + VariantEntry Entry; > > + Entry.Refs = Sym; > > + Yout << Entry; > > + } > > } > > > > Expected<IndexFileIn> readYAML(StringRef Data) { > > SymbolSlab::Builder Symbols; > > + RefSlab::Builder Refs; > > llvm::yaml::Input Yin(Data); > > do { > > - Symbol S; > > - Yin >> S; > > + VariantEntry Variant; > > + Yin >> Variant; > > if (Yin.error()) > > return llvm::errorCodeToError(Yin.error()); > > - Symbols.insert(S); > > + if (Variant.Symbol) > > + Symbols.insert(*Variant.Symbol); > > + if (Variant.Refs) > > + for (const auto &Ref : Variant.Refs->second) > > + Refs.insert(Variant.Refs->first, Ref); > > } while (Yin.nextDocument()); > > > > IndexFileIn Result; > > Result.Symbols.emplace(std::move(Symbols).build()); > > + Result.Refs.emplace(std::move(Refs).build()); > > return std::move(Result); > > } > > > > @@ -217,6 +290,17 @@ std::string toYAML(const Symbol &S) { > > } > > return Buf; > > } > > + > > +std::string toYAML(const std::pair<SymbolID, ArrayRef<Ref>> &Data) { > > + RefBundle Refs = {Data.first, Data.second}; > > + std::string Buf; > > + { > > + llvm::raw_string_ostream OS(Buf); > > + llvm::yaml::Output Yout(OS); > > + Yout << Refs; > > + } > > + return Buf; > > +} > > > > } // namespace clangd > > } // namespace clang > > > > Modified: clang-tools-extra/trunk/clangd/indexer/IndexerMain.cpp > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/clangd/indexer/IndexerMain.cpp?rev=343778&r1=343777&r2=3437 > > 78&view=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/clangd/indexer/IndexerMain.cpp (original) > > +++ clang-tools-extra/trunk/clangd/indexer/IndexerMain.cpp Thu Oct 4 > > 07:09:55 2018 > > @@ -67,18 +67,30 @@ public: > > else > > Symbols.insert(Sym); > > } > > + }, > > + [&](RefSlab S) { > > + std::lock_guard<std::mutex> Lock(SymbolsMu); > > + for (const auto &Sym : S) { > > + // No need to merge as currently all Refs are from > > main file. > > + for (const auto &Ref : Sym.second) > > + Refs.insert(Sym.first, Ref); > > + } > > }) > > .release(); > > } > > > > // Awkward: we write the result in the destructor, because the > > executor > > // takes ownership so it's the easiest way to get our data back out. > > - ~IndexActionFactory() { Result.Symbols = std::move(Symbols).build(); > > } > > + ~IndexActionFactory() { > > + Result.Symbols = std::move(Symbols).build(); > > + Result.Refs = std::move(Refs).build(); > > + } > > > > private: > > IndexFileIn &Result; > > std::mutex SymbolsMu; > > SymbolSlab::Builder Symbols; > > + RefSlab::Builder Refs; > > }; > > > > } // namespace > > > > Modified: clang-tools- > > extra/trunk/unittests/clangd/SerializationTests.cpp > > URL: http://llvm.org/viewvc/llvm-project/clang-tools- > > extra/trunk/unittests/clangd/SerializationTests.cpp?rev=343778&r1=34377 > > 7&r2=343778&view=diff > > ======================================================================= > > ======= > > --- clang-tools-extra/trunk/unittests/clangd/SerializationTests.cpp > > (original) > > +++ clang-tools-extra/trunk/unittests/clangd/SerializationTests.cpp Thu > > Oct 4 07:09:55 2018 > > @@ -13,6 +13,9 @@ > > #include "gmock/gmock.h" > > #include "gtest/gtest.h" > > > > +using testing::_; > > +using testing::AllOf; > > +using testing::Pair; > > using testing::UnorderedElementsAre; > > using testing::UnorderedElementsAreArray; > > namespace clang { > > @@ -21,6 +24,7 @@ namespace { > > > > const char *YAML = R"( > > --- > > +!Symbol > > ID: 057557CEBF6E6B2DD437FBF60CC58F352D1DF856 > > Name: 'Foo1' > > Scope: 'clang::' > > @@ -46,6 +50,7 @@ IncludeHeaders: > > References: 3 > > ... > > --- > > +!Symbol > > ID: 057557CEBF6E6B2DD437FBF60CC58F352D1DF858 > > Name: 'Foo2' > > Scope: 'clang::' > > @@ -64,6 +69,18 @@ Flags: 2 > > Signature: '-sig' > > CompletionSnippetSuffix: '-snippet' > > ... > > +!Refs > > +ID: 057557CEBF6E6B2DD437FBF60CC58F352D1DF856 > > +References: > > + - Kind: 4 > > + Location: > > + FileURI: file:///path/foo.cc > > + Start: > > + Line: 5 > > + Column: 3 > > + End: > > + Line: 5 > > + Column: 8 > > )"; > > > > MATCHER_P(ID, I, "") { return arg.ID == > > cantFail(SymbolID::fromStr(I)); } > > @@ -107,6 +124,16 @@ TEST(SerializationTest, YAMLConversions) > > EXPECT_EQ(Sym2.CanonicalDeclaration.FileURI, "file:///path/bar.h"); > > EXPECT_FALSE(Sym2.Flags & Symbol::IndexedForCodeCompletion); > > EXPECT_TRUE(Sym2.Flags & Symbol::Deprecated); > > + > > + ASSERT_TRUE(bool(ParsedYAML->Refs)); > > + EXPECT_THAT(*ParsedYAML->Refs, > > + UnorderedElementsAre( > > + Pair(cantFail(SymbolID::fromStr( > > + > > "057557CEBF6E6B2DD437FBF60CC58F352D1DF856")), > > + testing::SizeIs(1)))); > > + auto Ref1 = ParsedYAML->Refs->begin()->second.front(); > > + EXPECT_EQ(Ref1.Kind, RefKind::Reference); > > + EXPECT_EQ(Ref1.Location.FileURI, "file:///path/foo.cc"); > > } > > > > std::vector<std::string> YAMLFromSymbols(const SymbolSlab &Slab) { > > @@ -115,24 +142,37 @@ std::vector<std::string> YAMLFromSymbols > > Result.push_back(toYAML(Sym)); > > return Result; > > } > > +std::vector<std::string> YAMLFromRefs(const RefSlab &Slab) { > > + std::vector<std::string> Result; > > + for (const auto &Sym : Slab) > > + Result.push_back(toYAML(Sym)); > > + return Result; > > +} > > > > TEST(SerializationTest, BinaryConversions) { > > auto In = readIndexFile(YAML); > > EXPECT_TRUE(bool(In)) << In.takeError(); > > > > // Write to binary format, and parse again. > > - IndexFileOut Out; > > - Out.Symbols = In->Symbols.getPointer(); > > + IndexFileOut Out(*In); > > Out.Format = IndexFileFormat::RIFF; > > std::string Serialized = llvm::to_string(Out); > > + { > > + std::error_code EC; > > + llvm::raw_fd_ostream F("/tmp/foo", EC); > > + F << Serialized; > > + } > > > > auto In2 = readIndexFile(Serialized); > > ASSERT_TRUE(bool(In2)) << In.takeError(); > > - ASSERT_TRUE(In->Symbols); > > + ASSERT_TRUE(In2->Symbols); > > + ASSERT_TRUE(In2->Refs); > > > > // Assert the YAML serializations match, for nice comparisons and > > diffs. > > EXPECT_THAT(YAMLFromSymbols(*In2->Symbols), > > UnorderedElementsAreArray(YAMLFromSymbols(*In- > > >Symbols))); > > + EXPECT_THAT(YAMLFromRefs(*In2->Refs), > > + UnorderedElementsAreArray(YAMLFromRefs(*In->Refs))); > > } > > > > } // namespace > > > > > > _______________________________________________ > > cfe-commits mailing list > > cfe-commits@lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits >
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
