Re: OutOfMemoryError with FSDirectory

2011-04-05 Thread Michael McCandless
Try 1) reducing the RAM buffer of your IndexWriter (IndexWriter.setRAMBufferSizeMB), 2) using a term divisor when opening your reader (pass 2 or 3 or 4 as termInfosIndexDivisor when opening IndexReader), and 3) disabling norms or not indexing as many fields as possible. 70Mb is not that much RAM t

OutOfMemoryError with FSDirectory

2011-04-05 Thread Claudio R
Hi, I am using Lucene 2.9.4 with FSDirectory. My index has 80 thousand documents (each document has 12 fields). My jvm has 70Mb of RAM memory (limited by my hosting). I am getting various OutOfMemoryError. I ran jmap and I got: num   #instances    #bytes    Class description -

OutOfMemoryError with FSDirectory

2011-04-05 Thread Claudio R
Hi, I am using Lucene 2.9.4 with FSDirectory. My index has 80 thousand documents (each document has 12 fields). My jvm has 70Mb of RAM memory (limited by my hosting). I am getting various OutOfMemoryError. I ran jmap and I got: num   #instances    #bytes    Class description -

OutOfMemoryError with FSDirectory

2011-04-05 Thread Claudio R
Hi, I am using Lucene 2.9.4 with FSDirectory. My index has 80 thousand documents (each document has 12 fields). My jvm has 70Mb of RAM memory (limited by my hosting). I am getting various OutOfMemoryError. I ran jmap and I got: num   #instances    #bytes    Class description -

Re: OutOfMemoryError with FSDirectory

2011-04-04 Thread Claudio
Ok Erick, Thanks for your quick answer. FSDirectory will, indeed, store the index on disk. However, when *using* that index, lots of stuff happens. Specifically: When indexing, there is a buffer that accumulates documents until it's flushed to disk. Are you indexing? When searching (and this

Re: OutOfMemoryError with FSDirectory

2011-04-04 Thread Erick Erickson
FSDirectory will, indeed, store the index on disk. However, when *using* that index, lots of stuff happens. Specifically: When indexing, there is a buffer that accumulates documents until it's flushed to disk. Are you indexing? When searching (and this is the more important part), various caches a

OutOfMemoryError with FSDirectory

2011-04-04 Thread Claudio
Hi, I am using Lucene 2.9.4 with FSDirectory. My index has 80 thousand documents (each document has 12 fields). My jvm has 70Mb of RAM memory (limited by my hosting). I am getting various OutOfMemoryError. I ran jmap and I got: num #instances#bytesClass description