On 05/08/2011 17:10, Dante Bell wrote: > This is probably a really dumb question, but say they implement > load-balanced Tomcat on 2 nodes for example. Would that then allow for > greater than 20 STMs for the servlets?
It will allow them up to 20 concurrent requests per STM Servlet per Tomcat instance. What that means for actual users is difficult to judge since a single request may be routed through multiple servlets (including the same servlet several times). It will certainly increase capacity. From what to what is hard impossible to tell without knowing the application code. Mark > > On 08/05/2011 12:00 PM, Mark Thomas wrote: >> On 05/08/2011 16:56, Dante Bell wrote: >>> Thanks! >>> >>> Like I said, I'm an OS/HW guy, never looked at java b4! >>> >>> They are saying that the load test has 20 'connections' so I'm guessing >>> that's the 20 STMs. >>> >>> Now, is this a fixable thing within the Java stack? Or is it an >>> application limitation? >> It looks to be hard-coded within Tomcat. I don't see a way to change >> that limit without building Tomcat from source. >> >> The other option is re-write the STM Servlet(s) as non-STM. >> >> Mark >> >>> Danté >>> >>> On 08/05/2011 11:12 AM, Mark Thomas wrote: >>>> On 05/08/2011 15:34, Dante Bell wrote: >>>>> Hi, >>>>> >>>>> I'm running out of ideas on what to try for this customer. Their load >>>>> tests show that Tomcat is getting to a point where it no longer services >>>>> requests. >>>> Let me guess. It is fine for low loads but as soon as the load goes >>>> above a certain number (maybe around 20 concurrent users?) then >>>> everything starts going wrong? >>>> >>>>> Thread dumps show most threads in a wait state, but some are >>>>> runnable. I'm suspecting GC is the issue, >>>> On what basis? The thread dump clearly shows a different problem. >>>> >>>>> but in analyzing the logs it >>>>> shows the longest pause was 15 seconds, but that only happened once, and >>>>> most of the stats look OK to me. >>>> That would suggest that it isn't GC then wouldn't it. >>>> >>>>> Details can be found on my blog: http://wp.me/plPvN-ai >>>> All the information needed to diagnose this issue is in the thead dump. >>>> If you take a look at the first thread that is blocked: >>>> "TP-Processor40745" daemon prio=3 tid=0x03831400 nid=0xa888 in >>>> Object.wait() [0x2cd2f000..0x2cd2f870] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> at java.lang.Object.wait(Object.java:485) >>>> at >>>> org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:854) >>>> - locked <0x63a26708> (a java.util.Stack) >>>> at >>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:129) >>>> at >>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>>> >>>> And Bingo! we have found the problem. >>>> >>>> Now that stack trace might not ring alarm bells with someone unfamiliar >>>> with the Tomcat code base but if Tomcat is unresponsive, understanding >>>> why *any* thread is blocked would be a good place to start. If you look >>>> at line 854 and the surrounding code for StandardWrapper you will see >>>> that this is part of the Servlet allocation process. You should then >>>> realise that line 854 is part of Servlet allocation for Servlets that >>>> implement the SingleThreadModel (and now the alarm bells should be >>>> ringing loud and clear). >>>> >>>> Tomcat allocates a maximum of 20 instances of any STM servlet. Your >>>> client has an application that uses STM and requires many more than 20 >>>> concurrent instances. Hence most requests are sat waiting for a STM >>>> instance to be released. >>>> >>>> Since that means there must be 20 instances of an STM Servlet already >>>> allocated, it is simple enough to grep the thread dump to find out which >>>> one. The winner is: >>>> com.motorola.nsm.common.ui.servlet.ValidateServlet >>>> >>>> Mark >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
