Hey gang, The finally block was not hiding any exceptions: I'm still seeing the hangs in IndexWriter.close(). I'm appending the relevant output from 'setInfoStream(System.out)'. It looks like after I order the close, the IndexWriter begins a flush, and then hangs.
Any ideas? Thanks, Stu >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IFD [main]: setInfoStream [EMAIL PROTECTED] IW 0 [main]: setInfoStream: dir=org.apache.lucene.store.FSDirectory@/data1/solr/solr_dovecot1208772000-1208808000_f0cad01b1cb2966a/index autoCommit=true [EMAIL PROTECTED] [EMAIL PROTECTED] ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1 maxFieldLength=10000 index=_dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 IW 0 [main]: setMergeScheduler [EMAIL PROTECTED] >> Fetching >> hdfs://hadoop.sat.mlsrvr.com:54310/user/hadoopuser/indexed/dovecot/1208807515-1208808145-1208808184/part-00004 >> into a RAMDirectory: >> .. done. Size: 22053888 >> Merging directories to solr_dovecot1208772000-1208808000_f0cad01b1cb2966a >> (factor: 4): IW 0 [main]: flush at addIndexesNoOptimize IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 flushDocs=false flushDeletes=false flushDocStores=false numDocs=0 numBufDelTerms=0 IW 0 [main]: index before flush _dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 IW 0 [main]: now start transaction IW 0 [main]: flush at startTransaction IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 flushDocs=false flushDeletes=false flushDocStores=false numDocs=0 numBufDelTerms=0 IW 0 [main]: index before flush _dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 IW 0 [main]: LMP: findMerges: 20 segments IW 0 [main]: LMP: level 13.601146 to 14.351146: 2 segments IW 0 [main]: LMP: level 12.390487 to 13.140487: 3 segments IW 0 [main]: LMP: level 11.542906 to 12.292906: 3 segments IW 0 [main]: LMP: level 10.656786 to 11.406786: 2 segments IW 0 [main]: LMP: level -1.0 to 8.768208: 10 segments IW 0 [main]: LMP: 10 to 14: add this merge IW 0 [main]: LMP: 14 to 18: add this merge IW 0 [main]: add merge to pendingMerges: _1f:Cx1000->_0 _1g:Cx1000->_0 _1h:Cx1000->_0 _1i:Cx1000->_0 [total 1 pending] IW 0 [main]: add merge to pendingMerges: _1j:Cx1000->_0 _1k:Cx1000->_0 _1l:Cx1000->_0 _1m:Cx1000->_0 [total 2 pending] IW 0 [main]: now merge merge=_1f:Cx1000->_0 _1g:Cx1000->_0 _1h:Cx1000->_0 _1i:Cx1000->_0 into _mq index=_dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 _1e:Cx50000->_0 _1f:Cx1000->_0 _1g:Cx1000->_0 _1h:Cx1000->_0 _1i:Cx1000->_0 _1j:Cx1000->_0 _1k:Cx1000->_0 _1l:Cx1000->_0 _1m:Cx1000->_0 _1n:Cx1000->_0 _1o:Cx857->_0 IW 0 [main]: merging _1f:Cx1000->_0 _1g:Cx1000->_0 _1h:Cx1000->_0 _1i:Cx1000->_0 into _mq IW 0 [main]: merge: total 4000 docs IW 0 [main]: commitMerge _1f:Cx1000->_0 _1g:Cx1000->_0 _1h:Cx1000->_0 _1i:Cx1000->_0 into _mq IFD [main]: now checkpoint "segments_m" [17 segments ; isCommit = false] IW 0 [main]: LMP: findMerges: 17 segments IW 0 [main]: LMP: level 13.601146 to 14.351146: 2 segments IW 0 [main]: LMP: level 12.390487 to 13.140487: 3 segments IW 0 [main]: LMP: level 11.542906 to 12.292906: 3 segments IW 0 [main]: LMP: level 10.656786 to 11.406786: 2 segments IW 0 [main]: LMP: level -1.0 to 10.283832: 7 segments IW 0 [main]: LMP: 10 to 14: add this merge IW 0 [main]: now merge merge=_1j:Cx1000->_0 _1k:Cx1000->_0 _1l:Cx1000->_0 _1m:Cx1000->_0 into _mr index=_dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 _1e:Cx50000->_0 _mq:C4000 _1j:Cx1000->_0 _1k:Cx1000->_0 _1l:Cx1000->_0 _1m:Cx1000->_0 _1n:Cx1000->_0 _1o:Cx857->_0 IW 0 [main]: merging _1j:Cx1000->_0 _1k:Cx1000->_0 _1l:Cx1000->_0 _1m:Cx1000->_0 into _mr IW 0 [main]: merge: total 4000 docs IW 0 [main]: commitMerge _1j:Cx1000->_0 _1k:Cx1000->_0 _1l:Cx1000->_0 _1m:Cx1000->_0 into _mr IFD [main]: now checkpoint "segments_m" [14 segments ; isCommit = false] IW 0 [main]: LMP: findMerges: 14 segments IW 0 [main]: LMP: level 13.601146 to 14.351146: 2 segments IW 0 [main]: LMP: level 12.390487 to 13.140487: 3 segments IW 0 [main]: LMP: level 11.542906 to 12.292906: 3 segments IW 0 [main]: LMP: level 10.656786 to 11.406786: 2 segments IW 0 [main]: LMP: level -1.0 to 10.284627: 4 segments IW 0 [main]: LMP: 10 to 14: add this merge IW 0 [main]: add merge to pendingMerges: _mq:C4000 _mr:C4000 _1n:Cx1000->_0 _1o:Cx857->_0 [total 1 pending] IW 0 [main]: now merge merge=_mq:C4000 _mr:C4000 _1n:Cx1000->_0 _1o:Cx857->_0 into _ms index=_dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 _1e:Cx50000->_0 _mq:C4000 _mr:C4000 _1n:Cx1000->_0 _1o:Cx857->_0 IW 0 [main]: merging _mq:C4000 _mr:C4000 _1n:Cx1000->_0 _1o:Cx857->_0 into _ms IW 0 [main]: merge: total 9857 docs IW 0 [main]: commitMerge _mq:C4000 _mr:C4000 _1n:Cx1000->_0 _1o:Cx857->_0 into _ms IFD [main]: now checkpoint "segments_m" [11 segments ; isCommit = false] IFD [main]: delete "_mq.fnm" IFD [main]: delete "_mq.frq" IFD [main]: delete "_mq.prx" IFD [main]: delete "_mq.tis" IFD [main]: delete "_mq.tii" IFD [main]: delete "_mq.nrm" IFD [main]: delete "_mq.fdx" IFD [main]: delete "_mq.fdt" IFD [main]: delete "_mr.fnm" IFD [main]: delete "_mr.frq" IFD [main]: delete "_mr.prx" IFD [main]: delete "_mr.tis" IFD [main]: delete "_mr.tii" IFD [main]: delete "_mr.nrm" IFD [main]: delete "_mr.fdx" IFD [main]: delete "_mr.fdt" IW 0 [main]: LMP: findMerges: 11 segments IW 0 [main]: LMP: level 13.601146 to 14.351146: 2 segments IW 0 [main]: LMP: level 12.390487 to 13.140487: 3 segments IW 0 [main]: LMP: level 11.542906 to 12.292906: 3 segments IW 0 [main]: LMP: level 10.656786 to 11.406786: 3 segments IW 0 [main]: add merge to pendingMerges: _1e:Cx50000->_0 [total 1 pending] IW 0 [main]: now merge merge=_1e:Cx50000->_0 into _mt index=_dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 _1e:Cx50000->_0 _ms:C9857 IW 0 [main]: merging _1e:Cx50000->_0 into _mt IW 0 [main]: merge: total 50000 docs IW 0 [main]: commitMerge _1e:Cx50000->_0 into _mt IFD [main]: now checkpoint "segments_m" [11 segments ; isCommit = false] IW 0 [main]: LMP: findMerges: 11 segments IW 0 [main]: LMP: level 13.601146 to 14.351146: 2 segments IW 0 [main]: LMP: level 12.390487 to 13.140487: 3 segments IW 0 [main]: LMP: level 11.542906 to 12.292906: 5 segments IW 0 [main]: LMP: 5 to 9: add this merge IW 0 [main]: LMP: level 10.339036 to 10.916957: 1 segments IW 0 [main]: add merge to pendingMerges: _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 [total 1 pending] IW 0 [main]: now commit transaction IW 0 [main]: checkpoint: wrote segments file "segments_n" IFD [main]: now checkpoint "segments_n" [11 segments ; isCommit = true] IFD [main]: deleteCommits: now remove commit "segments_m" IFD [main]: delete "segments_m" >> ... done. >> Closing the target index... IW 0 [main]: now flush at close IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 flushDocs=false flushDeletes=false flushDocStores=false numDocs=0 numBufDelTerms=0 IW 0 [main]: index before flush _dn:C1349678 _j6:C1149270 _j5:C249194 _m4:C242243 _mf:C224430 _mk:C70616 _ml:C88 _mp:C50000 _mo:C9905 _mt:C50000 _ms:C9857 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -----Original Message----- From: Stu Hood <[EMAIL PROTECTED]> Sent: Sunday, April 20, 2008 3:41pm To: java-user@lucene.apache.org Subject: Re: Occasional Hang in IndexWriter.close() Hey Mike, Thanks for your input... the 'IndexWriter.close' call was actually in a 'finally' block around the merge code, without a 'catch', which I realized may have been hiding the exception (I didn't realize close would block if an exception had occurred). I've moved the close out of the finally block, so hopefully if an exception was being hidden, I'll see it and report back. Thanks, Stu -----Original Message----- From: Michael McCandless <[EMAIL PROTECTED]> Sent: Saturday, April 19, 2008 4:40am To: java-user@lucene.apache.org, [EMAIL PROTECTED] Subject: Re: Occasional Hang in IndexWriter.close() --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]