I am trying to get a clustered Tomcat 6.0.20 environment to operate in a high load environment. As part of the testing regime, we want to be able to run continuous high load against a QA setup - and validate that Tomcat operates continuously without a hiccup.
The SETUP: 1) Apache front-end (don't ask me why - I don't control this part) using AJP through to proxy to the Tomcat cluster. 2) 2 Tomcat hosts 'clustered' (although for the test, I'm only actually hitting one of the tomcats). Both are identically configured, 8-processor, 8GB systems running Tomcat 6.0.20. The Cluster is functioning proper (near as we can tell). The Tomcats have PermSize=1g, Xmx=6g, Xms=3g. 3) 2 "load generators" using JMeter to make requests against the Apache host. They are running 100 threads (each) to request a simple unauthenticated page. They are running 20 threads each to perform "registration" and "login" (the primary difference being that registration and login makes 5 requests per thread, and they end up creating a much larger 'session' object when they are done). 4) Watching the 'session count' using "/manager/html" on the Tomcat that we're hitting. The "problem" we're seeing: 1) Watching the 'session' count we see the number of sessions increase to about 450,000 over the course of about 40 minutes. (Of those, they lean about 120-to-1 for simple "empty" sessions created by the single-page requests.) The sessions for the single-page requests have an expire after time of '1 second'. Near as we can tell, none of these sessions expire while we are under load. 2) We watch the garbage collection activity by tailing the log file, and notice that after we reach the 450,000 sessions limit, the Tomcat starts to perform Full GCs. The Full GCs take about 20 seconds (even a little more), and there is about 2-5 seconds between the Full GCs, so essentially 80+% of the time is spent doing Full GCs, the Tomcat is effectively 'hung' at this point. 3) EXCEPT, if we stop the load generator (and nothing else is changed), immediately the session count on the Tomcat begins to drop - and Full GCs stop within a couple more cycles. If we wait ~3 minutes, the sessions drop down to only the sessions that haven't reached their inactivity timeout - and we can restart the load generators and go again for another 40 mins (or so). The question: Is there a setting we're missing that tells Tomcat to make session expiration a higher priority? -- Robin D. Wilson Director of Web Development KingsIsle Entertainment, Inc. CELL: 512-426-3929 DESK: 512-623-5913 www.KingsIsle.com --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
