RE: OutOfMemoryError while enumerating through reader.terms(fieldName)

[Ramana Jelda]

 
Hi,
I just debugged it closely.. Sorry I am getting OutOfMemoryError not because
of reader.terms()
But because of invoking QueryFilter.bits() method for each unique term.
I will try explain u with psuedo code.

 while(term != null){
       if(term.field().equals(name)){
          String termText = term.text();
          keys.addElement(termText);
       }else{
         break;
       }
      if(te.next()){
        term = te.term();
       }else{
      break;
      }
 }

for(Iterator iter = keys.iterator(); iter.hasNext();){
  String termText = (String) iter.next();
 TermQuery termQuery = new TermQuery(new Term(fieldName, termText));
   QueryFilter filter = new QueryFilter(termQuery);
   final BitSet bits;
   bits = filter.bits(ciaoReader.getIndexReader());
   BitSet pr = cache.put(termText, bits);
}
}

Second for loop which gets BitSet using QueryFilter is now throwing
OutOfMemoryError.

Any advise is relly welcome.

Thx,
Jelda



> -----Original Message-----
> From: Ramana Jelda [mailto:[EMAIL PROTECTED] 
> Sent: Tuesday, May 02, 2006 12:55 PM
> To: java-user@lucene.apache.org
> Subject: OutOfMemoryError while enumerating through 
> reader.terms(fieldName)
> 
> Hi,
> I am getting OutOfMemoryError , while enumerating through  
> TermEnum  after invoking reader.terms(fieldName).
> 
> Just to provide you more information, I have almost 10000 
> unique terms in field A. I can successfully enumerate around 
> 5000terms but later I am gettting OutOfMemoryError.
> 
> I set jvm max memory as 512MB , Ofcourse my index is bigger 
> than this memory around 1GB-2GB..
> How can I ask lucene to cleanup loaded index and traverse 
> through remaining terms.. It seems while enumerating memory 
> always grows in steps of some MBs.
> 
> Any help would be really appreciable.
> 
> Thanks in advance,
> Jelda
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]