-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Suvendu,
On 1/18/18 2:55 AM, Suvendu Sekhar Mondal wrote: > Hi Christopher, > > On Wed, Jan 17, 2018 at 10:41 PM, Christopher Schultz > <ch...@christopherschultz.net> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >> >> Suvendu, >> >> Which version of Tomcat? >> > > Tomcat version is 7.0.55 > >> On 1/17/18 8:20 AM, Suvendu Sekhar Mondal wrote: >>> I am seeing one issue. Under high load sporadically one web >>> service call fails with: "java.net.SocketTimeoutException: >>> Read timed out" after 60 Sec mark. >>> >>> In our app, httpd routes requests to 8888 then it get >>> redirected to 8889. >> >> Can you be very explicit about what you mean by "redirect", >> here? > > I was talking about auto redirection from 8888 to 8889 by > Catalina. > >>> Connector which listens on 8888 has connectionTimeout=20000 but >>> the one which listens to 8889 does not have any >>> connectionTimeout set explicitly. As per doc, default >>> connectionTimeout value 60 Sec will be set for connector which >>> listens on 8889 - and that is playing a role here. Do you think >>> this assumption is correct? >>> >>> Here are the connector properties: <Connector port="8888" >>> protocol="HTTP/1.1" connectionTimeout="20000" >>> redirectPort="8889" /> >> >> So you are probably expecting that all HTTP traffic to :8888 will >> be redirected to HTTPS over :8889? >> > > Yes. > >> How long does the redirect take? >> > > I do not know how to measure redirection time. Please let me know > if there any mechanism to trace it. > >> Under heavy load, you might be running out of threads depending >> upon a lot of factors, such as what kind of protocol and endpoint >> are actually being used. It looks like you are using the APR >> connector (due to SSLCertificateFile) which has some positive >> attributes with request to scalability while NIO would probably >> be the best choice if possible. >> >> It's got terrible performance, though, unless you use the >> OpenSSL provider (which requires Tomcat 8.5 or later). >> >>> <Connector port="8889" SSLCertificateFile="C:/mycer.cer" >>> SSLCertificateKeyFile="C:\mykey.key" SSLEnabled="true" >>> acceptCount="100" clientAuth="false" >>> disableUploadTimeout="true" enableLookups="false" >>> keystoreFile="conf/.keystore" maxHttpHeaderSize="8192" >>> maxSavePostSize="-1" maxThreads="200" minSpareThreads="20" >>> protocol="HTTP/1.1" scheme="https" secure="true" >>> sslProtocol="TLS1.2" >>> compressableMimeTypes="text/html,text/xml,text/plain,application/jso n" >>> >>> >> >>> compressionMinSize="2048" >>> compression="force" threadPriority="6" /> >> >> When you say "high load", what kind of load are you talking >> about, specifically? >> >> You may simply have too much traffic for your existing hardware >> to handle. No amount of configuration can fix that. - -chris > > We have 9 JVMs. As per current setting, per JVM max concurrency > can reach up-to 300(maxThreads="200"+acceptCount="100"). Issue was > reported when 2300 users on the system. So that's 2300 users across all 9 JVMs with the same configuration? Your cluster ought to be able to support 9 * 200 = 1800 simultaneous active users plus 9 * 100 = 900 queued in the OS's TCP wait queue. 2300 should be within your capabilities, but it might depend heavily upon how the load-balancer decided to allocate requests. If it sent all the users to the same node in the cluster, then you'll see bad performance even if you "should" be able to handle that much load. Where is the SocketTimeoutException being thrown? On the server, or on the client? If it's on the server, then it's the client's fault (or the load-balancer's fault) and it's not a problem at all with your server capacity. If it's happening on the client, it makes a lot more sense that the server might be overloaded. But that 60-second timeout would have to be on the client, not on the server side. So I'm a little confused as to what is going on, here. > So, I think it is not a case of thread exhaustion but something > else. Most possibly few JVMs were under stress due to uneven load > balancing(we have stickysession). This issue just came to me and > all resource usage data are gone. I have asked for re-run so that I > can analyze it properly. Let's see. :) It would be good to see what the session-load and the current-user (meaning in-process request) load was on each node in the cluster at the time, and which of them seems to be failing. > Back to my original question, in case of HTTP to HTTPS auto > redirection, Connector properties set for the later will be > finally applied to the request - is that a correct statement? The order of events goes like this: 1. Client connects to server via HTTP 2. Server redirects client to HTTPS port [1] 3. Client connects to server via HTTPS 2. Server responds appropriately[2] [1] here is the 20s connect timeout, which isn't actually a connection timeout... it's a timeout waiting for the client to present the HTTP request line before terminating the connection. [2] here is the 60s connect timeout [which isn't a connection timeout, of course]. Once that second connection is made (the HTTPS connection), the 20s connect-timeout is no longer applicable. Hope that helps, - -chris -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlpgtisdHGNocmlzQGNo cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFg9thAAhv8j+ioCuJD2Vr/e ovZEWkFKVhl1yWD3YTUX0KoTGot1MYp096jlU3BWaTBNaNxb9ay2U1ar4WLtRfky 9SFVaDEKykKN88yBCAuj6LCFtqJq5kxk8jGnbvjeN66TEaOOCQaGZNHu/JxPiCz4 UI82ynx5HFVM0fvhHublyv8+GdaZvjZa0mDYcp/1p0ySwqlyD3gy7eBk04z3JDan GZ9DLdshG/YtIeMT8VmBae565PKOs8WGaxMoj+/tnbnCmPNLu1CzgoBuQDwCu7vp mKQMYPCsnX4Hfi9RVoa8d8porOi2TxaJcpF3KFTujGSFebzcRHEZh1DHWP7ugfcb 3Zx2WAYD9Mbf0k3INoBROpfG/nDHtqgMLVfPRmBEuX11QluB7b1uP73XLfviyg7V JeEsCrb/8lXGC9j9xS9wc02ozLZb0KI/AdHSBiu673Q0r5IEMTgBM7pLiuIyMT5U 4o58c1VtkqGuhgf0vOCDyxYoSM4IOfK+DpOvWbvByOjhNVAKHJObgBB7koF8g+xQ bIsr9P30zUukE6Er01/jGyfhMCZJP5HWot7mDM2RFLpseqSu2g9NGLkFCJiNKRpO uBOXh8mCq/LPuAHRqSmNKnNOxWmQ6/5I0yaXRF85KP4J4cXRI2037+90Pp+37QsV qOtrMzDtuE0x/Wlp17ARe38fkHw= =5NhI -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
