Hello, I keep getting java.lang.OutOfMemoryError while doing search and would like to get some ideas to resolve the issue..
BACKGROUND: IBM AIX server with 8 GIG of memory. Lucene index contains 625 million documents with each having roughly 200 bytes of stored fields/data. Running Lucene IndexSearcher in JVM with -Xms3072m -Xmx3072m (more memory than that then the system kills the JVM). PROBLEM: I can search successfully search the index if I do not user sort or filtering. Although I need to invoke IndexReader with a termInfosIndexDivisor of at least 100 (IndexReader.open(dir,NoDeletionPolicy.INSTANCE,true,100) If I include a sort ( of a numeric field) or a numeric range filter, I get OutOfMemoryError no matter how big the termInfosIndexDivisor is. I've tried it as big as 100,000,000 and still got OutOfMemoryError. It seems I am running against hard limits of resources. Perhaps 8 GB of memory is just simply not enough to handle an index of 600 million documents. But before telling management that they must get more memory, I'd to see if there might be other ways to accomplish this. Thanks in advance. Jason
