Hi,
I am trying to use a lucene as a kind of key value store, but I encountered
some bad performance issues. When I try to add my data as documents to the
index I get an average write rate of 3 documents / second!! This seems to
me ridiculously slow and I guess I must have somewhere an error. Please
have a look at my code:
Directory dir = new [email protected]!
[email protected]!ry(file);
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_45);
IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_45,
analyzer);
IndexWriter writer = new IndexWriter(dir, config);
int eventCount = 1000;
for(int i=0; i < eventCount;i++){
Document doc = new Document();
doc.add(new StringField("id", i+"id" ,Store.YES));
doc.add(new StoredField("b", buildVector()));
writer.addDocument(doc);
writer.commit();
}
dir.close();
writer.close()
Not calling the commit function seems to fix the issue, but I guess this
would then have some issues if I want to read values in the mean time. My
normal use case would be to read something from the index, maybe alter it
and then write back. So I would have roughly 50% of reads.
I tried also an embedded version of elastic search and it manages to go to
2000 documents/ per second. As its based on lucene as well I guess I do
something wrong in my code.
THX for the help,
Klaus
--
--
Klaus Schaefers
Senior Optimization Manager
Ligatus GmbH
Hohenstaufenring 30-32
D-50674 Köln
Tel.: +49 (0) 221 / 56939 -784
Fax: +49 (0) 221 / 56 939 - 599
E-Mail: [email protected]
Web: www.ligatus.de
HRB Köln 56003
Geschäftsführung:
Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
Dipl.-Wirtschaftsingenieur Arne Wolter
