how to avoid OutOfMemoryError while indexing ?

2013-01-26 Thread wgggfiy
I found it is very easy to come into OutOfMemoryError. My idea is that lucene could set the RAM memory Automatically, but I couldn't find the API. My code: IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_40, analyzer); int mb = 1024 * 1024; double ram = Runtime.getRuntime().maxMemory

Re: MMapDirectory performance - Are searchable field values contiguously stored in FS block?

2013-01-26 Thread Michael McCandless
Hi Gili, I responded last time you asked this: http://lucene.markmail.org/thread/svun5cdtgiy4hnjg Maybe you are not subscribed to the list? Mike McCandless http://blog.mikemccandless.com On Sat, Jan 26, 2013 at 7:45 AM, Gili Nachum wrote: > Hi, > > I have a search workload that focuses o

MMapDirectory performance - Are searchable field values contiguously stored in FS block?

2013-01-26 Thread Gili Nachum
Hi, I have a search workload that focuses on two fields in my 1GB index. I get very good performance when loaded the index via MMapDirectory. I attribute this performance to the Operating System File System (FS OS) cache, that keeps the most recently used FS blocks RAM resident. *I would like to

Re: about isStored method

2013-01-26 Thread Andrew Gilmartin
I seems that you would benefit from reading at least one of the great books on Lucene. Lucene in Action is a good place to start. Until the, perhaps the following will be helpful Lin Ma wrote: - For stored field, it means its original information/structure is kept for search results (

Re: about isStored method

2013-01-26 Thread Lin Ma
Thanks Uwe, The information you provided is very informative. Two more questions/comments, - For stored field, it means its original information/structure is kept for search results (I think for an indexed field it means means lost its original form to indexed form), correct? - What m