Identified the problem.
reader.close() was not getting called in a specific logic flow.
Thank You.
Kumar _/|\_
www.saisk.com
ku...@saisk.com
"making a profound difference with knowledge and creativity..."
On Thu, Dec 31, 2009 at 11:11 AM, Kumaravel Kandasami <
kumaravel.kandas...@gmail.com> wrote:
> Thanks Mike.
>
> I think it is something to do with the merge factor.
> Modified the code to do optimize in the finally block the following error
> message was thrown.
>
> Code Snippet:
> nameWriter.optimize(); // errors here
> nameWriter.close();
> valueWriter.optimize(); //I am using multiple writers
> simultaneously. Writing to different directory.
> valueWriter.close()
>
>
> Index Directory Files:
> _1o.cfs
> _1p.cfs
> _1q.cfs
> _1r.cfs
> _1s.cfs
> _1t.fnm
> segments_1
> segments.gen
> write.lock
>
>
> Exception in thread "main" java.io.IOException: background merge hit
> exception: _1o:c55 _1p:c1 _1q:c1 _1r:c1 _1s:c1 into _1t [optimize]
> [mergeDocStores]
> at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2854)
> at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2789)
> at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2759)
> at
> com.hli.vis.index.VisualSoftIndexWriter.commitChanges(VSIndexWriter.java:46)
> at com.hli.vis.index.Main.main(Main.java:39)
> Caused by: java.io.FileNotFoundException:
> /Users/Kumar/Dev/Cust/index/names/_1t.fdx (Too many open files)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
> at
> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.<init>(SimpleFSDirectory.java:186)
> at
> org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:76)
> at
> org.apache.lucene.index.FieldsWriter.<init>(FieldsWriter.java:82)
> at
> org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:334)
> at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:153)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5029)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> Java Result: 1
>
>
>
> Kumar _/|\_
> www.saisk.com
> ku...@saisk.com
> "making a profound difference with knowledge and creativity..."
>
>
> On Thu, Dec 31, 2009 at 6:45 AM, Michael McCandless <
> luc...@mikemccandless.com> wrote:
>
>> It sounds like you may be running out of file descriptors -- how many
>> segments are in your index?
>>
>> The reopen logic looks correct (you are closing the old reader). Is
>> there anything else that may be holding files open?
>>
>> Have you changed any of IW's settings, eg mergeFactor?
>>
>> Mike
>>
>> On Wed, Dec 30, 2009 at 3:21 PM, Kumaravel Kandasami
>> <kumaravel.kandas...@gmail.com> wrote:
>> > I am getting IOException when I am doing a "Real-time" search, i.e. I am
>> > creating a Index using the Index Writer and also opening the Index using
>> > Index Reader (writer.getReader()) to make sure the document does not
>> exist
>> > prior adding to the Index file.
>> >
>> > The code works perfect fine multiple time indexed across couple of GB
>> data,
>> > however, on a TB it fails with the exception listed below.
>> > Any suggestions appreciated. Thanks. Kumar
>> >
>> > Code Snippet:
>> >
>> > IndexReader reader2 = reader.reopen();//get
>> >
>> > if (reader2 != reader) {
>> >
>> > reader.close();
>> > reader = reader2;
>> > searcher = new IndexSearcher(reader);
>> > reader2 = null;
>> > }
>> >
>> >
>> > Exception:
>> >
>> > Exception in thread "main" java.io.IOException: directory
>> > '/Users/Kumar/Dev/Cust/index/values' exists and is a directory, but
>> cannot
>> > be listed: list() returned null
>> > at
>> org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:516)
>> > at
>> org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:531)
>> > at
>> >
>> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:307)
>> > at
>> >
>> org.apache.lucene.index.IndexWriter.doFlushInternal(IndexWriter.java:4300)
>> > at
>> > org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:4192)
>> > at
>> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:4183)
>> > at
>> > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:470)
>> > at
>> > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:444)
>> > at
>> >
>> org.apache.lucene.index.DirectoryReader.doReopen(DirectoryReader.java:384)
>> > at
>> > org.apache.lucene.index.DirectoryReader.reopen(DirectoryReader.java:348)
>> >
>> > Kumar _/|\_
>> > www.saisk.com
>> > ku...@saisk.com
>> > "making a profound difference with knowledge and creativity..."
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>
>>
>
