-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Mitchell,
On 10/21/14 8:08 PM, Smith, Mitchell wrote: > Tomcat7 Java1.7_025 Thanks for adding the version numbers. How are you configuring Tomcat's thread pool? Either through one or more <Connector>s or with one or more <Executor>s. Can you post the (sanitized) configurations for any of those that are in CATALINA_BASE/conf/server.xml? > On 21 October 2014 20:06, Smith, Mitchell <mitchell.sm...@cwc.com> > wrote: > >> Hi >> >> I have an issue where my applicaion servers have recently >> started failing with OOM - cannot create native thread errors. That's the important part: "cannot create native thread". So, you haven't run out of memory, you have run out of threads, file descriptors, or stack space. The solution is to either reduce the number of threads in your JVM or to modify your process's limits to allow more resources to be used. The decision of which action to take depends upon your requirements... >> However the dump files all show he majority of threads (over >> 90%) like the following: >> >> "pool-21088-thread-1" - Thread t@36407 >>> >>> java.lang.Thread.State: WAITING at sun.misc.Unsafe.park(Native >>> Method) - parking to wait for <2bc1d4f0> (a >>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) >>> >>> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) >>> at >>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) >>> >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079) >>> >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> >>> at java.lang.Thread.run(Thread.java:724) >>> Locked ownable synchronizers: - None >>> >>> >>> >> My first assumption is that these are for a connection pool, >> however I am seeing close to 1000 (one thousand) and these never >> appear to timeout, or close. Yes, these are Tomcat's request-processor threads. 1000 seems like a lot. The default is 250 per connector so you must have modified your configuration. They never "time-out" or close because they are waiting for work to be done (e.g. for more requests to be handled). If you have many many idle worker threads, perhaps you should reduce the size of your thread pool. >> I could of course increase the number of threads the OS allows, >> but this of course has memory implications etc and is not really >> solving the root cause. The real question is whether or not you need all those threads at all. Are you finding that you have peak-load that your server can't handle? If not, try scaling-back the number of threads you actually use. Do you have any instrumentation in production to see how much traffic your server is actually handling? Do you know how many requests/min or req/sec you are handling? How about max concurrent requests during a certain period of time? >> Can anyone identify what these are? I see some posts online >> indicating these are the connection pool waiting for new >> requests. But this doesnt seem to match my configuation for >> acceptable connections Post your configuration, and we'll take a look. - -chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQIcBAEBCAAGBQJUR7ZFAAoJEBzwKT+lPKRYDJwP/R1wWgYUf94H3xHkl/F67LNh VQkceESfamZ9ADNo1jr9oSIyKmHfcy1St7ih8+izdfJSX8tDwbzN/QnFBIoTDrDM j8JkODHR0VKRa1Wyt34kPpwThx8ABRLIfh4jA/8+XDKv/57UPcOAGRpc6YzuaXNW h0IS82qoLsgjLrSFzH/cUGToFKVN1H2RhnMrWCRWNkcqosKIGdddf3VwNxE2gW9F D7vGmxfVUMff88y5craIENmrw7v1q8mQxhQCS0qeUFEzgQA8AVHlE54rk5sGbqXB ItKaMlpkQEllaz5h7FGpcSmA5jrHhy1SiC0gOAad21kmqgXOaiJdMtd91mDBqylE zxd5XGeh8TLSCeKFjRFox326TsovkHxtSonneWOTkTIb1qqpFGeFbdNDjpSMNnlC Q3atlo4hkEgh+QxCH+ntuCzG3ziXWyQuWuvqF4VyHxa9oX+8SkWRaOoqmcADyZbm bHSIcUHBpQ5UihK3j2ViChGF+DKvTMbFXUIRKjGbjS7ey6krkRXM9tJKGDlYWMVn JKDcvmE6aSFpXTehxFRAvPmaNNegneimEPM/QECtw1cAYQmqjUWwugHdM2Gb6rzH OZltocdoUYLRFBIIUwV9yVdJXhlmeFl2s/RvM31kr+ujoKA+zC5DqtUv5b08t9kM DAkZI5/Xtdo+fZ7xjFpV =m7XY -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
