This is probably a really dumb question, but say they implement load-balanced Tomcat on 2 nodes for example. Would that then allow for greater than 20 STMs for the servlets?
On 08/05/2011 12:00 PM, Mark Thomas wrote: > On 05/08/2011 16:56, Dante Bell wrote: >> Thanks! >> >> Like I said, I'm an OS/HW guy, never looked at java b4! >> >> They are saying that the load test has 20 'connections' so I'm guessing >> that's the 20 STMs. >> >> Now, is this a fixable thing within the Java stack? Or is it an >> application limitation? > It looks to be hard-coded within Tomcat. I don't see a way to change > that limit without building Tomcat from source. > > The other option is re-write the STM Servlet(s) as non-STM. > > Mark > >> Danté >> >> On 08/05/2011 11:12 AM, Mark Thomas wrote: >>> On 05/08/2011 15:34, Dante Bell wrote: >>>> Hi, >>>> >>>> I'm running out of ideas on what to try for this customer. Their load >>>> tests show that Tomcat is getting to a point where it no longer services >>>> requests. >>> Let me guess. It is fine for low loads but as soon as the load goes >>> above a certain number (maybe around 20 concurrent users?) then >>> everything starts going wrong? >>> >>>> Thread dumps show most threads in a wait state, but some are >>>> runnable. I'm suspecting GC is the issue, >>> On what basis? The thread dump clearly shows a different problem. >>> >>>> but in analyzing the logs it >>>> shows the longest pause was 15 seconds, but that only happened once, and >>>> most of the stats look OK to me. >>> That would suggest that it isn't GC then wouldn't it. >>> >>>> Details can be found on my blog: http://wp.me/plPvN-ai >>> All the information needed to diagnose this issue is in the thead dump. >>> If you take a look at the first thread that is blocked: >>> "TP-Processor40745" daemon prio=3 tid=0x03831400 nid=0xa888 in >>> Object.wait() [0x2cd2f000..0x2cd2f870] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> at java.lang.Object.wait(Object.java:485) >>> at >>> org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:854) >>> - locked <0x63a26708> (a java.util.Stack) >>> at >>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:129) >>> at >>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>> >>> And Bingo! we have found the problem. >>> >>> Now that stack trace might not ring alarm bells with someone unfamiliar >>> with the Tomcat code base but if Tomcat is unresponsive, understanding >>> why *any* thread is blocked would be a good place to start. If you look >>> at line 854 and the surrounding code for StandardWrapper you will see >>> that this is part of the Servlet allocation process. You should then >>> realise that line 854 is part of Servlet allocation for Servlets that >>> implement the SingleThreadModel (and now the alarm bells should be >>> ringing loud and clear). >>> >>> Tomcat allocates a maximum of 20 instances of any STM servlet. Your >>> client has an application that uses STM and requires many more than 20 >>> concurrent instances. Hence most requests are sat waiting for a STM >>> instance to be released. >>> >>> Since that means there must be 20 instances of an STM Servlet already >>> allocated, it is simple enough to grep the thread dump to find out which >>> one. The winner is: >>> com.motorola.nsm.common.ui.servlet.ValidateServlet >>> >>> Mark >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
