Tamara,
You may use StringBuffer instead of String docText =
hits.doc(j).getField("DOCUMENT").stringValue() ;
after that you may use StringBuffer.delete() to release memery.
Another way is using x64-bit machine.
Regards,
Mead
On Wed, Oct 19, 2011 at 5:14 AM, Otis Gospodnetic <
otis_gospodne...@yahoo.com> wrote:
> Bok Tamara,
>
> You didn't say what -Xmx value you are using. Try a little higher value.
> Note that loading field values (and it looks like this one may be big
> because is compressed) from a lot of hits is not recommended.
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
> >________________________________
> >From: Tamara Bobic <tamara.bo...@scai.fraunhofer.de>
> >To: java-user@lucene.apache.org
> >Cc: Roman Klinger <roman.klin...@scai.fraunhofer.de>
> >Sent: Tuesday, October 18, 2011 12:21 PM
> >Subject: OutOfMemoryError
> >
> >Hi all,
> >
> >I am using Lucene to query Medline abstracts and as a result I get around
> 3 million hits. Each of the hits is processed and information from a certain
> field is used.
> >
> >After certain number of hits, somewhere around 1 million (not always the
> same number) I get OutOfMemory exception that looks like this:
> >
> >Exception in thread "main" java.lang.OutOfMemoryError
> > at java.util.zip.Inflater.inflateBytes(Native Method)
> > at java.util.zip.Inflater.inflate(Inflater.java:221)
> > at java.util.zip.Inflater.inflate(Inflater.java:238)
> > at
> org.apache.lucene.document.CompressionTools.decompress(CompressionTools.java:108)
> > at
> org.apache.lucene.index.FieldsReader.uncompress(FieldsReader.java:609)
> > at
> org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:385)
> > at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:231)
> > at
> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:1013)
> > at
> org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:520)
> > at
> org.apache.lucene.index.FilterIndexReader.document(FilterIndexReader.java:149)
> > at org.apache.lucene.index.IndexReader.document(IndexReader.java:947)
> > at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:152)
> > at org.apache.lucene.search.MultiSearcher.doc(MultiSearcher.java:156)
> > at org.apache.lucene.search.Hits.doc(Hits.java:180)
> > at
> de.fhg.scai.bio.tamara.corpusBuilding.LuceneCmdLineInterface.queryMedline(LuceneCmdLineInterface.java:178)
> > at
> de.fhg.scai.bio.tamara.corpusBuilding.LuceneCmdLineInterface.main(LuceneCmdLineInterface.java:152)
> >
> >
> >this line which causes problems is:
> >String docText = hits.doc(j).getField("DOCUMENT").stringValue() ;
> >
> >I am using java 1.6 and I tried solving this issue with different garbage
> collectors (-XX:+UseParallelGC and -XX:+UseParallelOldGC) but it didn't
> help.
> >
> >Does anyone have any idea how to solve this problem?
> >
> >There is also an official bug report:
> >http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6293787
> >
> >Help is much appreciated. :)
> >
> >Best regards,
> >Tamara Bobic
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> >For additional commands, e-mail: java-user-h...@lucene.apache.org
> >
> >
> >
> >
>
