Re: OutofMemory in large index

2009-11-15 Thread vsevel
sort; wildcards from time to time. vincent Otis Gospodnetic wrote: > > Hello, > > > Comments inlined. > > > - Original Message >> To: java-user@lucene.apache.org >> Sent: Fri, November 13, 2009 11:32:02 AM >> Subject: Re: OutofMemory in la

Re: OutofMemory in large index

2009-11-13 Thread Otis Gospodnetic
Hello, Comments inlined. - Original Message > From: vsevel > To: java-user@lucene.apache.org > Sent: Fri, November 13, 2009 11:32:02 AM > Subject: Re: OutofMemory in large index > > > Hi, I am jumping into the thread because I have got a similar issue. > My

Re: OutofMemory in large index

2009-11-13 Thread vsevel
Hi, I am jumping into the thread because I have got a similar issue. My index is 30Gb large and contains 21M docs. I was able to stay with 1Gb of RAM on the server for a while. Recently I started to simulate parallel searches. Just 2 parallel searches would get the server to crash with out of memo

Re: OutofMemory in large index

2009-11-13 Thread Simon Willnauer
On Fri, Nov 13, 2009 at 12:01 PM, Wenbo Zhao wrote: > Thank you all...  I think I need to read more docs > > A little question : how to add more memory over 1G ? > When I specify more than -Xmx1450M, jvm gives error: >>java -Xmx1450m asdf > Exception in thread "main" java.lang.NoClassDefFoundError

Re: OutofMemory in large index

2009-11-13 Thread Wenbo Zhao
Thank you all... I think I need to read more docs A little question : how to add more memory over 1G ? When I specify more than -Xmx1450M, jvm gives error: >java -Xmx1450m asdf Exception in thread "main" java.lang.NoClassDefFoundError: asdf >java -Xmx1451m asdf Error occurred during initializati

Re: OutofMemory in large index

2009-11-13 Thread Michael McCandless
Interrupting optimize shouldn't cause any problems. It should have no effect on the index, except possibly the partially created files might be orphan'd (left on disk but not referenced by the index), in which case they'll be cleaned up the next time you open a writer on the index. Still, running

Re: OutofMemory in large index

2009-11-13 Thread Simon Willnauer
On Fri, Nov 13, 2009 at 11:17 AM, Ian Lea wrote: >> I got OutOfMemoryError at >> org.apache.lucene.search.Searcher.search(Searcher.java:183) >> My index is 43G bytes.  Is that too big for Lucene ? >> Luke can see the index has over 1800M docs, but the search is also out >> of memory. >> I use -Xmx

Re: OutofMemory in large index

2009-11-13 Thread Ian Lea
> I got OutOfMemoryError at > org.apache.lucene.search.Searcher.search(Searcher.java:183) > My index is 43G bytes.  Is that too big for Lucene ? > Luke can see the index has over 1800M docs, but the search is also out > of memory. > I use -Xmx1024M to specify 1G java heap space. 43Gb is not too bi