Re: Lucene: Index Writer to write in multiple file instead make one heavy file

2014-05-15 Thread Michael McCandless
Just don't call optimize... In theory, you could make a custom Directory impl that would split a single large file (from Lucene's standpoint) into multiple OS files, but this ... would be a lot of work. It's simpler to just not optimize. Mike McCandless http://blog.mikemccandless.com On Wed,

Re: Lucene: Index Writer to write in multiple file instead make one heavy file

2014-05-15 Thread Yogesh patel
In my case, it creates CFS(Compound File) As 10 GB. Can we split that file while optimize or writing index. Thanks On Wed, May 14, 2014 at 7:38 PM, Yogesh patel wrote: > Thanks for reply!!! > > Can you please provide me sample code for it? I got the idea but i dont > know how to impleme

Re: Lucene: Index Writer to write in multiple file instead make one heavy file

2014-05-14 Thread Yogesh patel
Thanks for reply!!! Can you please provide me sample code for it? I got the idea but i dont know how to implement it. Thanks On Tue, May 13, 2014 at 7:02 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > You can tell the MergePolicy to limit the maximum size of segments it >

RE: Lucene: Index Writer to write in multiple file instead make one heavy file

2014-05-13 Thread Toke Eskildsen
Yogesh patel [yogeshpateldai...@gmail.com] wrote: > I am using lucene 3.0.1. I am writing many documents with lucene > Indexwriter. But Indexwriter add all documents into file which becomes more > than 4GB in my case. so can i distribute files or partitioned ? Normally Lucene does not produce a si

Re: Lucene: Index Writer to write in multiple file instead make one heavy file

2014-05-13 Thread Michael McCandless
You can tell the MergePolicy to limit the maximum size of segments it should merge. Also, you should try to upgrade: 3.0.1 is REALLY old. Mike McCandless http://blog.mikemccandless.com On Tue, May 13, 2014 at 1:58 AM, Yogesh patel wrote: > HI > > I am using lucene 3.0.1. I am writing many doc

Lucene: Index Writer to write in multiple file instead make one heavy file

2014-05-12 Thread Yogesh patel
HI I am using lucene 3.0.1. I am writing many documents with lucene Indexwriter. But Indexwriter add all documents into file which becomes more than 4GB in my case. so can i distribute files or partitioned ? -- *Regards,Yogesh Patel*