Hi All,
We Have used Lucene as our Search Engine and all our applications
are deployed onto tomcat and running with thread pool size of 200.
Java Version - 1.6.0-rc
Lucene Version - 2.3.2
Tomcat Version - 6.0.14
OS - Red Hat Enterprise Linux ES release 4 (Nahant Update 5)
kernel - 2.6.9-55.0.2.ELsmp
RAM - 4 GB
Tomcat Memory - 1.5 GB
Index Size - 2 GB
After 10-12 hrs of tomcat running, tomcat
becomes irresponsive. After doing core dump of tomcat process We observed
that all tomcat threads are blocked (Thread-pool size-200). none of the
tomcat threads are in runnable state.
each thread at the time of the core dump is in BLOCKED state...The following
are the stack trace of blocked.
"MultiSearcher thread #3" daemon prio=10 tid=0x337ddc00 nid=0x4827 waiting
for monitor entry [0x2f2f0000..0x2f2f0ea0]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:235)
- waiting to lock <0x45d49d88> (a
org.apache.lucene.store.FSDirectory$FSIndexInput)
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76)
at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:63)
at
org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:123)
at
org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:154)
at
org.apache.lucene.index.TermInfosReader.scanEnum(TermInfosReader.java:223)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:217)
at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:54)
at org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:668)
at
org.apache.lucene.search.ConstantScoreTermQuery$TermWeight.scorer(ConstantScoreTermQuery.java:63)
at
org.apache.lucene.search.VBooleanQuery$BooleanWeight.scorer(VBooleanQuery.java:276)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:232)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:143)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:124)
at
org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:250)
"http-8080-194" daemon prio=10 tid=0x08927800 nid=0x128d waiting for monitor
entry [0x2e188000..0x2e189e20]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:235)
- waiting to lock <0x45d49d88> (a
org.apache.lucene.store.FSDirectory$FSIndexInput)
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readVLong(IndexInput.java:96)
at
org.apache.lucene.index.MultiLevelSkipListReader.loadSkipLevels(MultiLevelSkipListReader.java:196)
at
org.apache.lucene.index.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:97)
at
org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDocs.java:164)
at in.verse.search.query.spans.TermSpans.skipTo(TermSpans.java:85)
at in.verse.search.query.spans.SpanScorer.skipTo(SpanScorer.java:70)
at
org.apache.lucene.search.VConjunctionScorer.doNext(VConjunctionScorer.java:78)
at
org.apache.lucene.search.VConjunctionScorer.next(VConjunctionScorer.java:71)
at
org.apache.lucene.search.VBooleanScorer2.next(VBooleanScorer2.java:456)
at
org.apache.lucene.search.VConjunctionScorer.init(VConjunctionScorer.java:136)
at
org.apache.lucene.search.VConjunctionScorer.next(VConjunctionScorer.java:65)
at
org.apache.lucene.search.VBooleanScorer2.score(VBooleanScorer2.java:412)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
at
org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:173)
at org.apache.lucene.search.Searcher.search(Searcher.java:118)
at org.apache.lucene.search.Searcher.search(Searcher.java:97)
at VSearcher$VerticalSearcher$1.call(VSearcher.java:500)
at VSearcher$VerticalSearcher$1.call(VSearcher.java:491)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1737)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at
java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:200)
at
in.verse.search.VSearcher$VerticalSearcher.searchInternal(VSearcher.java:542)
at
in.verse.search.VSearcher$VerticalSearcher.searchInternal(VSearcher.java:476)
at in.verse.search.VSearcher$VerticalSearcher.search(VSearcher.java:384)
at in.verse.search.VSearcher.search(VSearcher.java:187)
at
in.verse.search.impl.SearchQuery.executeSearchQuery(SearchQuery.java:166)
at
in.verse.search.impl.SearchQuery.executeSearchQuery(SearchQuery.java:255)
at
org.apache.jsp.jobs.mobile.airtel.xhtml.search_jsp._jspService(search_jsp.java:869)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:393)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
at
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:852)
at
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:584)
at
org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1508)
at java.lang.Thread.run(Thread.java:619)
--
View this message in context:
http://www.nabble.com/Tomcat-Threads-are-BLOCKED-after-some-time-tp22308385p22308385.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
