Re: When does a segment file gets written...

2014-11-05 Thread Shouvik Bardhan
Ours is an index for keeps and also keeps growing for many weeks till we decide to re-ingest again. I am counting docs getting added within my little server and every 5 million docs or so I am calling commit. I will play with the threshold and also bring in an elapsed time till last commit to refin

Re: When does a segment file gets written...

2014-11-04 Thread Michael McCandless
On Tue, Nov 4, 2014 at 11:44 AM, Shouvik Bardhan wrote: > Thanks for the reply (and thanks for everything else too !!) Mike. You're welcome! > I am unable to understand when to call commit. Should I start counting the > number of documents I am ingesting and then say every 10 million docs do a

Re: When does a segment file gets written...

2014-11-04 Thread Shouvik Bardhan
Thanks for the reply (and thanks for everything else too !!) Mike. I am unable to understand when to call commit. Should I start counting the number of documents I am ingesting and then say every 10 million docs do a commit()? I dont want to do a commit too frequently cause that does not sound cor

Re: When does a segment file gets written...

2014-11-04 Thread Michael McCandless
Call commit. Mike McCandless http://blog.mikemccandless.com On Tue, Nov 4, 2014 at 8:28 AM, Shouvik Bardhan wrote: > Apologies if this is trivial but I have read what I could on the net and > could not come up with a good answer. > > I have stood up a lucene server (not solr) which listens on

When does a segment file gets written...

2014-11-04 Thread Shouvik Bardhan
Apologies if this is trivial but I have read what I could on the net and could not come up with a good answer. I have stood up a lucene server (not solr) which listens on 2 ports - one for indexing and one for search. Also on different threads. The indexer threads use one IndexWriter and the searc