I assume it must be memory related issue.Not sure just a guess. -Manish On Fri, Jul 3, 2009 at 3:42 PM, Carl Austin <carl.aus...@detica.com> wrote:
> Hi, > > I have an issue when optimizing a large index (40GB). Basically the > optimize falls over with an IOException stating there are not enough > system resources, within a RandomAccessFile.readBytes call followed back > to the CompoundFileWriter close method. I assume this is when creating a > new CFS file from the separate other files. > Unfortunately there is no hint at which system resource, and I have no > access to the machine this fails on, otherwise I would run some testing > to see what resource usage is going on. > The process running this, has 1.3GB of memory allowance. > > If anyone has any insights they would be much appreciated. Stack trace > included now: > > Exception in thread "Lucene Merge Thread #0" > org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: > Insufficient system resources exist to complete the requested service > at > org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(Co > ncurrentMergeScheduler.java:309) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr > entMergeScheduler.java:286) > Caused by: java.io.IOException: Insufficient system resources exist to > complete the requested service > at java.io.RandomAccessFile.readBytes(Native Method) > at java.io.RandomAccessFile.read(Unknown Source) > at > org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector > y.java:596) > at > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput. > java:136) > at > org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j > ava:228) > at > org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java > :184) > at > org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j > ava:204) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260) > at > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge > Scheduler.java:205) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr > entMergeScheduler.java:260) > Exception in thread "Thread-12957887" java.lang.RuntimeException: > java.io.IOException: background merge hit exception: _cx:c3072736 > _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791 _dn:c4045 _do:c3643 > _dp:c3197 into _dq [optimize] > at a.b.c.d.e.f.ImportThread$ImportRunner.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > Caused by: java.io.IOException: background merge hit exception: > _cx:c3072736 _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791 > _dn:c4045 _do:c3643 _dp:c3197 into _dq [optimize] > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2258) > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2203) > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2183) > ... 2 more > Caused by: java.io.IOException: Insufficient system resources exist to > complete the requested service > at java.io.RandomAccessFile.readBytes(Native Method) > at java.io.RandomAccessFile.read(Unknown Source) > at > org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector > y.java:596) > at > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput. > java:136) > at > org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j > ava:228) > at > org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java > :184) > at > org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j > ava:204) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260) > at > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge > Scheduler.java:205) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr > entMergeScheduler.java:260) > > Thanks > > Carl Austin > > > > > > This message should be regarded as confidential. If you have received this > email in error please notify the sender and destroy it immediately. > Statements of intent shall only become binding when confirmed in hard copy > by an authorised signatory. The contents of this email may relate to > dealings with other companies within the Detica Group Limited group of > companies. > > Detica Group Limited is registered in England under No: 03328242. > > Registered offices: Warwick House, PO Box 87, Farnborough Aerospace Centre, > Farnborough, Hampshire, GU14 6YU, England. > > -- Manish Joshi Adserving Team Zedo India
