Chris, As you described I have added two new connectors in server.xml and using nginx to redirect requests to different connector ports. Also configured nginx to route traffic of each app on a different connector port of tomcat
In config of each app I am using port specific for the app which is being called localhost:8081 for app2 and localhost:8082 for app 3 So now in config of app1 we call app2 using localhost:8081/app2 and localhost:8082/app3 Could you explain the benefit of using this type of config? Will this be useful to not block requests for each app? On Mon, 1 Jun 2020, 16:27 Christopher Schultz, <ch...@christopherschultz.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Ayub, > > On 5/31/20 09:20, Ayub Khan wrote: > > On single tomcat instance how to map each app to different port > > number? > > You'd have to use multiple <Engine> elements, which means separate > everything and not just the <Connector>. It's more work on the Tomcat > side with the same problem of having a different port number which you > can get just by using a separate <Connector>. > > Since you have a reverse-proxy already, it's simpler to use the > reverse-proxy as the port-selector and not worry about trying to > actually enforce it at the Tomcat level. > > - -chris > > > On Sun, 31 May 2020, 15:44 Christopher Schultz, < > > ch...@christopherschultz.net> wrote: > > > > Ayub, > > > > On 5/29/20 20:23, Ayub Khan wrote: > >>>> Chris, > >>>> > >>>> You might want (2) and (3) to have their own, independent > >>>> connector and thread pool, just to be safe. You don't want a > >>>> connection in (1) to stall because a loopback connection > >>>> can't be made to (2)/(3). Meanwhile, it's sitting there > >>>> making no progress but also consuming a connection+thread. > >>>> > >>>> *there is only one connector per tomcat where all the > >>>> applications > > receive > >>>> the requests, they do not have independent connector and > >>>> thread pool per tomcat. How to configure independent > >>>> connector and thread pool per application per tomcat instance > >>>> ? below is the current connector > > config in > >>>> each tomcat instance:* > > > > You can't allocate a connector to a particular web application -- > > at least not in the way that you think. > > > > What you have to do if use different port numbers. Users will > > never use them, though. But since you have nginx (finally! A reason > > to have it!), you can map /app1 to port 8080 and /app2 to port 8081 > > and /app3 to port 8083 or whatever you want. > > > > Internal loopback connections will either have to go through nginx > > (which I wouldn't recommend) or know the correct port numbers to > > use (which I *do* recommend). > > > > -chris > > > >>>> *<Connector port="8080" > >>>> protocol="org.apache.coyote.http11.Http11NioProtocol" > >>>> connectionTimeout="20000" URIEncoding="UTF-8" > >>>> redirectPort="8443" />* > >>>> > >>>> > >>>> > >>>> On Fri, May 29, 2020 at 9:05 PM Christopher Schultz < > >>>> ch...@christopherschultz.net> wrote: > >>>> > >>>> Ayub, > >>>> > >>>> On 5/28/20 17:25, Ayub Khan wrote: > >>>>>>> Nginx is being used for image caching and converting > >>>>>>> https to http requests before hitting tomcat. > >>>> So you encrypt between the ALB and your app server nodes? > >>>> That's fine, though nginx probably won't offer any > >>>> performance improvement for images (unless it's really > >>>> caching dynamically-generated images from your application) > >>>> or TLS termination. > >>>> > >>>>>>> The behavior I am noticing is application first throws > >>>>>>> Borken pipe client abort exception at random apis > >>>>>>> calls followed by socket timeout and then database > >>>>>>> connection leak errors. This happens only during high > >>>>>>> load. > >>>> > >>>> If you are leaking connections, that's going to be an > >>>> application resource-management problem. Definitely solve > >>>> that, but it shouldn't affect anything with Tomcat > >>>> connections and/or threads. > >>>> > >>>>>>> During normal traffic open files for tomcat process > >>>>>>> goes up and down to not more than 500. However during > >>>>>>> high traffic if I keep track of the open files for each > >>>>>>> tomcat process as soon as the open files count reaches > >>>>>>> above 10k that tomcat instance stops serving the > >>>>>>> requests. > >>>> > >>>> Any other errors shown in the logs? Like OutOfMemoryError > >>>> (for e.g. open files)? > >>>> > >>>>>>> If the open file count goes beyond 5k its sure that > >>>>>>> this number will never come back to below 500 at this > >>>>>>> point we need to restart tomcat. > >>>>>>> > >>>>>>> > >>>>>>> There are three application installed on each tomcat > >>>>>>> instance, > >>>>>>> > >>>>>>> 1) portal: portal calls (2) and (3) using localhost, > >>>>>>> should we change this to use dns name instead of > >>>>>>> localhost calls ? > >>>>>>> > >>>>>>> 2) Services for portal 3) Services for portal and > >>>>>>> mobile clients > >>>> > >>>> Are they all sharing the same connector / thread pool? > >>>> > >>>> You might want (2) and (3) to have their own, independent > >>>> connector and thread pool, just to be safe. You don't want a > >>>> connection in (1) to stall because a loopback connection > >>>> can't be made to (2)/(3). Meanwhile, it's sitting there > >>>> making no progress but also consuming a connection+thread. > >>>> > >>>> -chris > >>>> > >>>>>>> On Thu, May 28, 2020 at 4:50 PM Christopher Schultz < > >>>>>>> ch...@christopherschultz.net> wrote: > >>>>>>> > >>>>>>> Ayub, > >>>>>>> > >>>>>>> On 5/27/20 19:43, Ayub Khan wrote: > >>>>>>>>>> If we have 18 core CPU and 100GB RAM. What value > >>>>>>>>>> can I set for maxConnections ? > >>>>>>> Your CPU and RAM really have nothing to do with it. > >>>>>>> It's more about your usage profile. > >>>>>>> > >>>>>>> For example, if you are serving small static files, you > >>>>>>> can serve a million requests a minute on a Raspberry > >>>>>>> Pi, many of them concurrently. > >>>>>>> > >>>>>>> But if you are performing fluid dynamic simulations > >>>>>>> with each request, you will obviously need more > >>>>>>> horsepower to service a single request, let alone > >>>>>>> thousands of concurrent requests. > >>>>>>> > >>>>>>> If you have tons of CPU and memory to spare, feel free > >>>>>>> to crank-up the max connections. The default is 10000 > >>>>>>> which is fairly high. At some point, you will run out > >>>>>>> of connection allocation space in the OS's TCP/IP > >>>>>>> stack, so that is really your upper-limit. You simply > >>>>>>> cannot have more than the OS will allow. See > >>>>>>> https://stackoverflow.com/a/2332756/276232 for some > >>>>>>> information about that. > >>>>>>> > >>>>>>> Once you adjust your settings, perform a load-test. You > >>>>>>> may find that adding more resources actually slows > >>>>>>> things down. > >>>>>>> > >>>>>>>>>> Want to make sure we are utilizing the hardware > >>>>>>>>>> to the max capacity. Is there any config of > >>>>>>>>>> tomcat which enabled could help serve more > >>>>>>>>>> requests per tomcat instance. > >>>>>>> > >>>>>>> Not really. Improving performance usually come down to > >>>>>>> tuning the application to make the requests take less > >>>>>>> time to process. Tomcat is rarely the source of > >>>>>>> performance problems (but /sometimes/ is, and it's > >>>>>>> usually a bug that can be fixed). > >>>>>>> > >>>>>>> You can improve throughput somewhat by pipelineing > >>>>>>> requests. That means HTTP keepalive for direct > >>>>>>> connections (but with a small timeout; you don't want > >>>>>>> clients who aren't making any follow-up requests to > >>>>>>> waste your resources waiting for a keep-alive timeout > >>>>>>> to close a connection). For proxy connections (e.g. > >>>>>>> from nginx), you'll want those connections to remain > >>>>>>> open as long as possible to avoid the re-negotiation of > >>>>>>> TCP and possibly TLS handshakes. Using HTTP/2 can be > >>>>>>> helpful for performance, at the cost of some CPU on the > >>>>>>> back-end to perform the complicated connection > >>>>>>> management that h2 requires. > >>>>>>> > >>>>>>> Eliminating useless buffering is often very helpful. > >>>>>>> That's why I asked about nginx. What are you using it > >>>>>>> for, other than as a barrier between the load-balancer > >>>>>>> and your Tomcat instances? If you remove nginx, I > >>>>>>> suspect you'll see a measurable performance increase. > >>>>>>> This isn't a knock against nginx: you'd see a > >>>>>>> performance improvement by removing *any* reverse-proxy > >>>>>>> that isn't providing any value. But you haven't said > >>>>>>> anything about why it's there in the first place, so I > >>>>>>> don't know if it /is/ providing any value to you. > >>>>>>> > >>>>>>>>>> The current setup is able to handle most of the > >>>>>>>>>> load, however there are predictable times where > >>>>>>>>>> there is an avalanche of requests and thinking > >>>>>>>>>> how to handle it gracefully. > >>>>>>> > >>>>>>> You are using AWS: use auto-scaling. That's what it's > >>>>>>> for. > >>>>>>> > >>>>>>> -chris > >>>>>>> > >>>>>>>>>> On Wed, May 27, 2020 at 5:38 PM Christopher > >>>>>>>>>> Schultz < ch...@christopherschultz.net> wrote: > >>>>>>>>>> > >>>>>>>>>> Ayub, > >>>>>>>>>> > >>>>>>>>>> On 5/27/20 09:26, Ayub Khan wrote: > >>>>>>>>>>>>> previously I was using HTTP/1.1 connector, > >>>>>>>>>>>>> recently I changed to NIO2 to see the > >>>>>>>>>>>>> performance. I read that NIO2 is non > >>>>>>>>>>>>> blocking so trying to check how this > >>>>>>>>>>>>> works. > >>>>>>>>>> > >>>>>>>>>> Both NIO and NIO2 are non-blocking. They use > >>>>>>>>>> different strategies for certain things. Anything > >>>>>>>>>> but the "BIO" connector will be non-blocking for > >>>>>>>>>> most operations. The default is NIO. > >>>>>>>>>> > >>>>>>>>>>>>> which connector protocol do you recommend > >>>>>>>>>>>>> and best configuration for the connector ? > >>>>>>>>>> This depends on your environment, usage profile, > >>>>>>>>>> etc. Note that non-blocking IO means more CPU > >>>>>>>>>> usage: there is a trade-off. > >>>>>>>>>> > >>>>>>>>>>>>> Which stable version of tomcat would you > >>>>>>>>>>>>> recommend ? > >>>>>>>>>> > >>>>>>>>>> Always the latest, of course. Tomcat 8.0 is > >>>>>>>>>> unsupported, replaced by Tomcat 8.5. Tomcat 9.0 > >>>>>>>>>> is stable and probably the best version if you > >>>>>>>>>> are looking to upgrade. Both Tomcat 8.5 and 9.0 > >>>>>>>>>> are continuing to get regular updates. But > >>>>>>>>>> definitely move away from 8.0. > >>>>>>>>>> > >>>>>>>>>>>>> Are there any ubuntu specific configs for > >>>>>>>>>>>>> tomcat ? > >>>>>>>>>> No. There is nothing particular special about > >>>>>>>>>> Ubuntu. Linux is one of the most well-performing > >>>>>>>>>> platforms for the JVM. I wouldn't recommend > >>>>>>>>>> switching platforms. > >>>>>>>>>> > >>>>>>>>>> Why are you using nginx? You already have > >>>>>>>>>> load-balancing happening in the ALB. Inserting > >>>>>>>>>> another layer of proxying is probably just adding > >>>>>>>>>> another buffer to the mix. I'd remove nginx if > >>>>>>>>>> it's not providing any specific, measurable > >>>>>>>>>> benefit. > >>>>>>>>>> > >>>>>>>>>>>>> We are using OkHttp client library to call > >>>>>>>>>>>>> rest api and stack trace shows failure at > >>>>>>>>>>>>> the api call. The api being called is > >>>>>>>>>>>>> running on the same tomcat instance > >>>>>>>>>>>>> (different context) usring url localhost. > >>>>>>>>>>>>> This does not happen when number of > >>>>>>>>>>>>> requests is less. > >>>>>>>>>> > >>>>>>>>>> Your Tomcat server is calling this REST API? Or > >>>>>>>>>> your server is serving those API requests? If > >>>>>>>>>> your service is calling itself, then you have to > >>>>>>>>>> make sure you have double-capacity: every > >>>>>>>>>> incoming request will cause a loopback request to > >>>>>>>>>> your own service. > >>>>>>>>>> > >>>>>>>>>> Other than the timeouts, are you able to handle > >>>>>>>>>> the load with your existing infrastructure? > >>>>>>>>>> Sometimes, the solution is simply to throw most > >>>>>>>>>> hardware at the problem. > >>>>>>>>>> > >>>>>>>>>> -chris > >>>>>>>>>> > >>>>>>>>>>>>> On Wed, May 27, 2020 at 11:48 AM Mark > >>>>>>>>>>>>> Thomas <ma...@apache.org> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On 26/05/2020 23:28, Ayub Khan wrote: > >>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> During high load I am seeing below > >>>>>>>>>>>>>>> error on tomcat logs > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> java.util.concurrent.ExecutionException: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > java.net > >>>>>>>>>>>>>> .SocketTimeoutException: > >>>>>>>>>>>>>>> timeout > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> And the rest of that stack trace? It is > >>>>>>>>>>>>>> hard to provide advice without context. > >>>>>>>>>>>>>> We need to know what is timing out when > >>>>>>>>>>>>>> trying to do what. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We have 4 C5.18x large vms running > >>>>>>>>>>>>>>> tomcat 8 behind AWS application load > >>>>>>>>>>>>>>> balancer. We are seeing socket timeouts > >>>>>>>>>>>>>>> during peak hours. What should be the > >>>>>>>>>>>>>>> configuration of tomcat if we get > >>>>>>>>>>>>>>> 60,000 to 70,000 requests per > >>>>>>>>>>>>>> minute > >>>>>>>>>>>>>>> on an average ? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Tomcat 8.0.32 on Ubuntu 16.04.5 LTS > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Tomcat 8.0.x is no longer supported. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Below is the java version: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> java version "1.8.0_181" Java(TM) SE > >>>>>>>>>>>>>>> Runtime Environment (build > >>>>>>>>>>>>>>> 1.8.0_181-b13) Java HotSpot(TM) 64-Bit > >>>>>>>>>>>>>>> Server VM (build 25.181-b13, mixed > >>>>>>>>>>>>>>> mode) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Below is the server.xml connector > >>>>>>>>>>>>>>> configuration: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> <Connector port="8080" > >>>>>>>>>>>>>>> protocol="org.apache.coyote.http11.Http11Nio2Protocol" > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>> > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> > Why NIO2? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Mark > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> connectionTimeout="20000" > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> URIEncoding="UTF-8" redirectPort="8443" > >>>>>>>>>>>>>>> /> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We have 4 C5.18x large vms and each vm > >>>>>>>>>>>>>>> has nginx and tomcat instance running. > >>>>>>>>>>>>>>> All the 4 vms are connected to AWS > >>>>>>>>>>>>>>> application load balancer. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I tried to add maxConnections=50000 but > >>>>>>>>>>>>>>> this does not seem to have any affect > >>>>>>>>>>>>>>> and still saw the timeouts > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks and Regards Ayub > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> --------------------------------------------------------- > - --- > > > >>>>>>>>>>>>>> > - --- > >>>> > >>>>>>>>>>>>>> > > --- > >>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>> --- > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>> To unsubscribe, e-mail: > >>>>>>> users-unsubscr...@tomcat.apache.org > >>>>>>>>>>>>>> For additional commands, e-mail: > >>>>>>>>>>>>>> users-h...@tomcat.apache.org > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ------------------------------------------------------------ > - --- > > > >>>>>>>>>>> > - --- > >>>> > >>>>>>>>>>> > > --- > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>>>>>>>>>> For additional commands, e-mail: > >>>>>>>>>>> users-h...@tomcat.apache.org > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> --------------------------------------------------------------- > - --- > > > >>>>>>>> > - --- > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>> > >>>>>>>> > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>>>>>>> For additional commands, e-mail: > >>>>>>>> users-h...@tomcat.apache.org > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> > >>>>> > >>>>> ------------------------------------------------------------------ > - --- > >>>>> > >>>>> > > > >>>>> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>>>> For additional commands, e-mail: > >>>>> users-h...@tomcat.apache.org > >>>>> > >>>>> > >>>> > >> > >> --------------------------------------------------------------------- > >> > >> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > >> > > > -----BEGIN PGP SIGNATURE----- > Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ > > iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7VAhQACgkQHPApP6U8 > pFjsLxAAnoJRxvynux8z1JpmC1H3BGJ1rNPmKflhkibEMmbInEprkQtz80AHcyvJ > 3n3/itLD0VekxHMxUyIl3urHRqZh2MwxSRksITjyYXxok4R/UOW6noO2vCNiY1G8 > hn+EYGe2y5DTpBwh1Utk6Wy0bm8KD+xGP09IoMp4fmx3DNnL99mL9KbSxl9qJHSs > G0YwAK2uTYx6IkFPUpjVso48H2B8C/8YsJYS13WJEnEoPK7xQRyKPPtoCFSD6WEE > vY4OZQdLorK0FRLEBeXh1Bqv6BUeLf7DYKB2oB4ph4PBi7Dnlb3od8pnMUAEzOVf > 9jTZQnOdpZPMlA0yI6yzI4e819OETT4IpLUQHx98LojD9CQsH6RR05WftrcqqJE1 > hnwEmBaIsSIzaXJm/qtN/juxnKqMwMeUMTNBMuB8OMR3HrFQuV/zlwK89cQfSy6E > Sa3lFyarJyyLkgTXgEtLzKItiz5YqjekPZBDM8UNjXjE8dyt78nxMBSZhRj+cz/T > fGzT6CCf1PkSmoJ6g0TZ1sNm2A4aqiVDL4i1TinTWfPz2IpYJOIe9JaddTE14Fu2 > oC/TtFuKINM4flreTT+30TOIo0ZGEsplNLoyZWmAnYtEoeR2Sb0uBYVIlco+X+sq > UAIW1fldxReljlAdnTGp+nrmizH+NMiCIbHDzoQmc82jNFjm3SU= > =wBrc > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
