Re: Cost of keeping around IndexReader instances

2013-11-22 Thread Vitaly Funstein
UPDATE: I went with method 1, i.e. keeping IndexReader instances open between requests. Which brings me back to the original questions - is there any way of quantifying the impact of not closing a particular IndexReader? Does this depend on # of segments per index, open file count etc? On Thu, Oc

Re: Lucene multithreaded indexing problems

2013-11-22 Thread Uwe Schindler
Hi, Don't use addDocuments. This method is more made for so called block indexing (where all documents need to be on a block for block joins). Call addDocument for each document possibly from many threads. By this Lucene can better handle multithreading and free memory early. There is really no

Re: Lucene multithreaded indexing problems

2013-11-22 Thread Igor Shalyminov
- uwe@ Thanks Uwe! I changed the logic so that my workers only parse input docs into Documents, and indexWriter does addDocuments() by itself for the chunks of 100 Documents. Unfortunately, this behaviour reproduces: memory usage slightly increases with the number of processed documents, and at

Re: JLemmaGen project

2013-11-22 Thread Zuhra
Hi, Thanks for providing us with open source tool. I am interested about the Macedonian version. I can't test it in Eclipse, so I want to ask you is this version complete? Can we use it already in our project? Are there any problems I should consider when integrating with other projects? Thanks