-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Ayub,
On 5/27/20 19:43, Ayub Khan wrote: > If we have 18 core CPU and 100GB RAM. What value can I set for > maxConnections ? Your CPU and RAM really have nothing to do with it. It's more about your usage profile. For example, if you are serving small static files, you can serve a million requests a minute on a Raspberry Pi, many of them concurrently. But if you are performing fluid dynamic simulations with each request, you will obviously need more horsepower to service a single request, let alone thousands of concurrent requests. If you have tons of CPU and memory to spare, feel free to crank-up the max connections. The default is 10000 which is fairly high. At some point, you will run out of connection allocation space in the OS's TCP/IP stack, so that is really your upper-limit. You simply cannot have more than the OS will allow. See https://stackoverflow.com/a/2332756/276232 for some information about that. Once you adjust your settings, perform a load-test. You may find that adding more resources actually slows things down. > Want to make sure we are utilizing the hardware to the max > capacity. Is there any config of tomcat which enabled could help > serve more requests per tomcat instance. Not really. Improving performance usually come down to tuning the application to make the requests take less time to process. Tomcat is rarely the source of performance problems (but /sometimes/ is, and it's usually a bug that can be fixed). You can improve throughput somewhat by pipelineing requests. That means HTTP keepalive for direct connections (but with a small timeout; you don't want clients who aren't making any follow-up requests to waste your resources waiting for a keep-alive timeout to close a connection). For proxy connections (e.g. from nginx), you'll want those connections to remain open as long as possible to avoid the re-negotiation of TCP and possibly TLS handshakes. Using HTTP/2 can be helpful for performance, at the cost of some CPU on the back-end to perform the complicated connection management that h2 requires. Eliminating useless buffering is often very helpful. That's why I asked about nginx. What are you using it for, other than as a barrier between the load-balancer and your Tomcat instances? If you remove nginx, I suspect you'll see a measurable performance increase. This isn't a knock against nginx: you'd see a performance improvement by removing *any* reverse-proxy that isn't providing any value. But you haven't said anything about why it's there in the first place, so I don't know if it /is/ providing any value to you. > The current setup is able to handle most of the load, however > there are predictable times where there is an avalanche of requests > and thinking how to handle it gracefully. You are using AWS: use auto-scaling. That's what it's for. - -chris > On Wed, May 27, 2020 at 5:38 PM Christopher Schultz < > ch...@christopherschultz.net> wrote: > > Ayub, > > On 5/27/20 09:26, Ayub Khan wrote: >>>> previously I was using HTTP/1.1 connector, recently I >>>> changed to NIO2 to see the performance. I read that NIO2 is >>>> non blocking so trying to check how this works. > > Both NIO and NIO2 are non-blocking. They use different strategies > for certain things. Anything but the "BIO" connector will be > non-blocking for most operations. The default is NIO. > >>>> which connector protocol do you recommend and best >>>> configuration for the connector ? > This depends on your environment, usage profile, etc. Note that > non-blocking IO means more CPU usage: there is a trade-off. > >>>> Which stable version of tomcat would you recommend ? > > Always the latest, of course. Tomcat 8.0 is unsupported, replaced > by Tomcat 8.5. Tomcat 9.0 is stable and probably the best version > if you are looking to upgrade. Both Tomcat 8.5 and 9.0 are > continuing to get regular updates. But definitely move away from > 8.0. > >>>> Are there any ubuntu specific configs for tomcat ? > No. There is nothing particular special about Ubuntu. Linux is one > of the most well-performing platforms for the JVM. I wouldn't > recommend switching platforms. > > Why are you using nginx? You already have load-balancing happening > in the ALB. Inserting another layer of proxying is probably just > adding another buffer to the mix. I'd remove nginx if it's not > providing any specific, measurable benefit. > >>>> We are using OkHttp client library to call rest api and >>>> stack trace shows failure at the api call. The api being >>>> called is running on the same tomcat instance (different >>>> context) usring url localhost. This does not happen when >>>> number of requests is less. > > Your Tomcat server is calling this REST API? Or your server is > serving those API requests? If your service is calling itself, > then you have to make sure you have double-capacity: every > incoming request will cause a loopback request to your own > service. > > Other than the timeouts, are you able to handle the load with your > existing infrastructure? Sometimes, the solution is simply to > throw most hardware at the problem. > > -chris > >>>> On Wed, May 27, 2020 at 11:48 AM Mark Thomas >>>> <ma...@apache.org> wrote: >>>> >>>>> On 26/05/2020 23:28, Ayub Khan wrote: >>>>>> Hi, >>>>>> >>>>>> During high load I am seeing below error on tomcat logs >>>>>> >>>>>> java.util.concurrent.ExecutionException: java.net >>>>> .SocketTimeoutException: >>>>>> timeout >>>>> >>>>> And the rest of that stack trace? It is hard to provide >>>>> advice without context. We need to know what is timing out >>>>> when trying to do what. >>>>> >>>>>> We have 4 C5.18x large vms running tomcat 8 behind AWS >>>>>> application load balancer. We are seeing socket timeouts >>>>>> during peak hours. What should be the configuration of >>>>>> tomcat if we get 60,000 to 70,000 requests per >>>>> minute >>>>>> on an average ? >>>>>> >>>>>> Tomcat 8.0.32 on Ubuntu 16.04.5 LTS >>>>> >>>>> Tomcat 8.0.x is no longer supported. >>>>> >>>>>> Below is the java version: >>>>>> >>>>>> java version "1.8.0_181" Java(TM) SE Runtime Environment >>>>>> (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM >>>>>> (build 25.181-b13, mixed mode) >>>>>> >>>>>> >>>>>> Below is the server.xml connector configuration: >>>>>> >>>>>> <Connector port="8080" >>>>>> protocol="org.apache.coyote.http11.Http11Nio2Protocol" >>>>> >>>>> Why NIO2? >>>>> >>>>> Mark >>>>> >>>>> >>>>>> connectionTimeout="20000" >>>>>> >>>>>> URIEncoding="UTF-8" redirectPort="8443" /> >>>>>> >>>>>> We have 4 C5.18x large vms and each vm has nginx and >>>>>> tomcat instance running. All the 4 vms are connected to >>>>>> AWS application load balancer. >>>>>> >>>>>> I tried to add maxConnections=50000 but this does not >>>>>> seem to have any affect and still saw the timeouts >>>>>> >>>>>> Thanks and Regards Ayub >>>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------ - --- >>>>> >>>>> > >>>>> >>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>>> For additional commands, e-mail: >>>>> users-h...@tomcat.apache.org >>>>> >>>>> >>>> >> >> --------------------------------------------------------------------- >> >> >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7PwX0ACgkQHPApP6U8 pFhX9RAAyfSvUt/rU4xist6W2c8wcvC6dg7qQ1WhQ0yl68Jm7PNkVTffpNQzIh6t vD6TUvIckF1O0AeWIwRXpfgFPUc0DUBiyrkEPwnQsiNMjkhMtq4iuESiFb5i/2NK 0J3JUHYusIwjycmfXeRrkVIG9ei9339bklclzHc3+DZpgERF7FQ+JvhloDb+uv9h NOmKwxXHri7Ikcqt8vSbkPadYsmjTPnj+eGjcS5mYSi9/W2OoVLxz1NB32iUoIi0 3Y0L78DvddbPYBaJ7WzkxiqKcTBfhBhFeV2PJKY8WqLUQXJsDc5SIVESh++TIeCx gD0OPjnP8XTgyUp4BTJnP604SAI+egIRzmLDKJYlv+KKb6y1GqQgPUuiMcLwFmb5 7T+h6R805bZJAdnp8ToM+g8FJR4/cZcp8uwR7zLPeUYq8KNPPWG8pUGk7byg7h8J 8gRCQ75j5+jqjKtweEyncBi0rKS6QN73SKh1crCVJOy/1QymsD/9COyChfFximPh rAb9S35rcxzGq+sEF6d/TcIvQ/Zso6fVA29QXgKBUwW1QS8GQu9CY/odYDchBlVA Kv1KsDPWMFPPraVvnckDk8KKs9dwRCtIn9QzX2MgOGhIFK4TJbs7+PsAnchHWMLh dhyPx+HDygCKvx33UyXtZNEitqxx1FVJa+W6zPxwKCOT7ZQS2gQ= =PIuS -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
