Thanks Jigar I haven't tweaked ConcurrentMergeStrategy between 4.5.0 and 6.6. Here is what I have:
<mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"> <int name="maxThreadCount">30</int> <int name="maxMergeCount">30</int> </mergeScheduler> On Mon, Jul 31, 2017 at 8:56 PM, Jigar Shah <jigaronl...@gmail.com> wrote: > I faced such problem when I used nomergepolicy, and did some code to manual > merging segments which had bug and I had same issue. Make sure you have > default AFAIR ConcurrentMergeStrategy enabled. And its is configured with > appropriate settings. > > On Jul 31, 2017 11:21 PM, "Erick Erickson" <erickerick...@gmail.com> > wrote: > > > No, nothing's changed fundamentally. But you say: > > > > "We have some batch indexing scripts, which > > flood the solr servers with indexing requests (while keeping > open-searcher > > false)" > > > > What is your commit interval? Regardless of whether openSearcher is false > > or not, background merging continues apace with every commit. By any > chance > > did you change your merge policy (or not copy the one from 4x to 6x)? > Shot > > in the dark... > > > > Best, > > Erick > > > > On Mon, Jul 31, 2017 at 7:15 PM, Nawab Zada Asad Iqbal <khi...@gmail.com > > > > wrote: > > > Hi, > > > > > > I am upgrading from solr4.5 to solr6.6 and hitting this issue during > > > complete reindexing scenario. We have some batch indexing scripts, > which > > > flood the solr servers with indexing requests (while keeping > > open-searcher > > > false) for many hours and then perform one commit. This used to work > fine > > > with 4.5, but with 6.6, i get 'Too many open files' within a couple of > > > minutes. I have checked that "ulimit" is same between old and new > > servers. > > > > > > Has something fundamentally changed in recent lucene versions, which > > keeps > > > file descriptors around for a longer time? > > > > > > > > > Here is a sample error message: > > > at org.apache.lucene.index.IndexWriter.ensureOpen( > > IndexWriter.java:749) > > > at org.apache.lucene.index.IndexWriter.ensureOpen( > > IndexWriter.java:763) > > > at org.apache.lucene.index.IndexWriter.commit( > IndexWriter.java:3206) > > > at > > > org.apache.solr.update.DirectUpdateHandler2.commit( > > DirectUpdateHandler2.java:644) > > > at > > > org.apache.solr.update.processor.RunUpdateProcessor.processCommit( > > RunUpdateProcessorFactory.java:93) > > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processCommit( > > UpdateRequestProcessor.java:68) > > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor. > > doLocalCommit(DistributedUpdateProcessor.java:1894) > > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor. > > processCommit(DistributedUpdateProcessor.java:1871) > > > at > > > org.apache.solr.update.processor.LogUpdateProcessorFactory$ > > LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160) > > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processCommit( > > UpdateRequestProcessor.java:68) > > > at > > > org.apache.solr.handler.RequestHandlerUtils.handleCommit( > > RequestHandlerUtils.java:68) > > > at > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody( > > ContentStreamHandlerBase.java:62) > > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest( > > RequestHandlerBase.java:173) > > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) > > > at org.apache.solr.servlet.HttpSolrCall.execute( > > HttpSolrCall.java:723) > > > at org.apache.solr.servlet.HttpSolrCall.call( > HttpSolrCall.java:529) > > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter( > > SolrDispatchFilter.java:361) > > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter( > > SolrDispatchFilter.java:305) > > > at > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain. > > doFilter(ServletHandler.java:1691) > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle( > > ServletHandler.java:582) > > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle( > > ScopedHandler.java:143) > > > at > > > org.eclipse.jetty.security.SecurityHandler.handle( > > SecurityHandler.java:548) > > > at > > > org.eclipse.jetty.server.session.SessionHandler. > > doHandle(SessionHandler.java:226) > > > at > > > org.eclipse.jetty.server.handler.ContextHandler. > > doHandle(ContextHandler.java:1180) > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doScope( > > ServletHandler.java:512) > > > at > > > org.eclipse.jetty.server.session.SessionHandler. > > doScope(SessionHandler.java:185) > > > at > > > org.eclipse.jetty.server.handler.ContextHandler. > > doScope(ContextHandler.java:1112) > > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle( > > ScopedHandler.java:141) > > > at > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle( > > ContextHandlerCollection.java:213) > > > at > > > org.eclipse.jetty.server.handler.HandlerCollection. > > handle(HandlerCollection.java:119) > > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle( > > HandlerWrapper.java:134) > > > at > > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle( > > RewriteHandler.java:335) > > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle( > > HandlerWrapper.java:134) > > > at org.eclipse.jetty.server.Server.handle(Server.java:534) > > > at org.eclipse.jetty.server.HttpChannel.handle( > HttpChannel.java:320) > > > at > > > org.eclipse.jetty.server.HttpConnection.onFillable( > > HttpConnection.java:251) > > > at > > > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded( > > AbstractConnection.java:273) > > > at org.eclipse.jetty.io.FillInterest.fillable( > FillInterest.java:95) > > > at > > > org.eclipse.jetty.io.SelectChannelEndPoint$2.run( > > SelectChannelEndPoint.java:93) > > > at > > > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume. > > executeProduceConsume(ExecuteProduceConsume.java:303) > > > at > > > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume. > > produceConsume(ExecuteProduceConsume.java:148) > > > at > > > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run( > > ExecuteProduceConsume.java:136) > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( > > QueuedThreadPool.java:671) > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run( > > QueuedThreadPool.java:589) > > > at java.lang.Thread.run(Thread.java:748) > > > Caused by: java.nio.file.FileSystemException: > > > /local/var/solr/shard2/filesearch/data/index/_34w5.fdx: Too many open > > files > > > at > > > sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) > > > at sun.nio.fs.UnixException.rethrowAsIOException( > > UnixException.java:102) > > > at sun.nio.fs.UnixException.rethrowAsIOException( > > UnixException.java:107) > > > at > > > sun.nio.fs.UnixFileSystemProvider.newByteChannel( > > UnixFileSystemProvider.java:214) > > > at > > > java.nio.file.spi.FileSystemProvider.newOutputStream( > > FileSystemProvider.java:434) > > > at java.nio.file.Files.newOutputStream(Files.java:216) > > > at > > > org.apache.lucene.store.FSDirectory$FSIndexOutput.< > > init>(FSDirectory.java:413) > > > at > > > org.apache.lucene.store.FSDirectory$FSIndexOutput.< > > init>(FSDirectory.java:409) > > > at > > > org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253) > > > at > > > org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput( > > LockValidatingDirectoryWrapper.java:44) > > > at > > > org.apache.lucene.store.TrackingDirectoryWrapper.createOutput( > > TrackingDirectoryWrapper.java:43) > > > at > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter. > > <init>(CompressingStoredFieldsWriter.java:114) > > > at > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat. > > fieldsWriter(CompressingStoredFieldsFormat.java:128) > > > at > > > org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat. > > fieldsWriter(Lucene50StoredFieldsFormat.java:183) > > > at > > > org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter( > > StoredFieldsConsumer.java:39) > > > at > > > org.apache.lucene.index.StoredFieldsConsumer.startDocument( > > StoredFieldsConsumer.java:46) > > > at > > > org.apache.lucene.index.DefaultIndexingChain.startStoredFields( > > DefaultIndexingChain.java:364) > > > at > > > org.apache.lucene.index.DefaultIndexingChain.processDocument( > > DefaultIndexingChain.java:398) > > > at > > > org.apache.lucene.index.DocumentsWriterPerThread.updateDocument( > > DocumentsWriterPerThread.java:232) > > > at > > > org.apache.lucene.index.DocumentsWriter.updateDocument( > > DocumentsWriter.java:478) > > > at > > > org.apache.lucene.index.IndexWriter.updateDocument( > > IndexWriter.java:1571) > > > at > > > org.apache.solr.update.DirectUpdateHandler2.updateDocument( > > DirectUpdateHandler2.java:924) > > > at > > > org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues( > > DirectUpdateHandler2.java:913) > > > at > > > org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate( > > DirectUpdateHandler2.java:302) > > > at > > > org.apache.solr.update.DirectUpdateHandler2.addDoc0( > > DirectUpdateHandler2.java:239) > > > at > > > org.apache.solr.update.DirectUpdateHandler2.addDoc( > > DirectUpdateHandler2.java:194) > > > at > > > org.apache.solr.update.processor.RunUpdateProcessor.processAdd( > > RunUpdateProcessorFactory.java:67) > > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd( > > UpdateRequestProcessor.java:55) > > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor. > versionAdd( > > DistributedUpdateProcessor.java:1005) > > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor. > processAdd( > > DistributedUpdateProcessor.java:748) > > > at > > > org.apache.solr.update.processor.LogUpdateProcessorFactory$ > > LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103) > > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd( > > UpdateRequestProcessor.java:55) > > > at > > > org.apache.solr.update.processor.LanguageIdentifierUpdateProces > > sor.processAdd(LanguageIdentifierUpdateProcessor.java:205) > > > at > > > org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java: > > 261) > > > at org.apache.solr.handler.loader.XMLLoader.load( > XMLLoader.java:188) > > > at > > > org.apache.solr.handler.UpdateRequestHandler$1.load( > > UpdateRequestHandler.java:97) > > > at > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody( > > ContentStreamHandlerBase.java:68) > > > ... 33 more > > > > > > > > > > > > Thanks > > > Nawab > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > >
