On 08/01/2019 23:51, Mason Meier wrote: > Hello, > > I'm running Tomcat-8.5 with TLS and I've noticed substantial memory growth > with requests over time, to the point that if I run Tomcat in Docker and > make constant requests to it, Docker will kill the container due to > excessive memory utilization. The problem occurs with standalone Tomcat as > well. Over the course of millions of requests, the memory usage of the > Tomcat process grows continuously, seemingly without bound.
I've behaviour like this in the past. From memory there was some caching in the TLS implementation at the root of most of it that could be controlled with some system properties. It may be you are seeing the same thing. Or you may have found a memory leak. The next step would be to use a profiler to see where the memory is being used. > I've done a fair amount of testing on AWS EC2 instances and some local > machines, and here are my observations: > * 'org.apache.tomcat.util.net.openssl.OpenSSLImplementation' seems to > increase memory utilization more quickly and consistently than > 'org.apache.tomcat.util.net.jsse.JSSEImplementation'. The > JSSEImplementation doesn't cause the memory to grow in certain setups. Can you share some configs that demonstrate an issue and some that don't. That might help narrow down what is going on. Mark > * Limiting the heap size of the JVM does not affect the amount of memory > consumed > * Other than decreasing a small amount (presumably due to garbage > collection), memory utilization stays consistent after the flow of requests > is stopped > > My testing consists of: > 1. Installing brand new versions of everything > 2. Generating a certificate > 3. Changing my server.xml to the one shown below > 4. Replacing 'webapps/ROOT/index.jsp' with a shorter payload > 5. Starting Tomcat > 6. Making millions of parallel requests from a different computer and > watching Tomcat's memory utilization grow > > Is there anything in my 'server.xml' that would be cause for concern? Are > there any known memory leaks in Tomcat's OpenSSL implementation? What steps > can I take to debug this problem? > > Versions: > Tomcat - apache-tomcat-8.5.37 > Java - JDK-1.8u191 > OpenSSL - openssl-1.0.2q > APR - apr-1.6.5 > Tomcat Native - tomcat-native-1.2.19 > OS - Amazon Linux release 2 (Karoo) > uname -a - Linux 4.14.77-81.59.amzn2.x86_64 #1 SMP Mon Nov 12 21:32:48 UTC > 2018 x86_64 x86_64 x86_64 GNU/Linux > > Here is my 'server.xml' file: > <?xml version="1.0" encoding="UTF-8"?> > <Server port="8005" shutdown="SHUTDOWN"> > <Listener className="org.apache.catalina.startup.VersionLoggerListener" /> > <Listener className="org.apache.catalina.core.AprLifecycleListener" > SSLEngine="on" /> > <Listener > className="org.apache.catalina.core.JreMemoryLeakPreventionListener" /> > <Listener > className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" /> > <Listener > className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" /> > > <Service name="Catalina"> > <Connector protocol="org.apache.coyote.http11.Http11NioProtocol" > > > sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation" > port="8443" > SSLEnabled="true" scheme="https" secure="true" > keystoreFile="server.keystore" keystorePass="<REDACTED>" > clientAuth="optional" sslProtocol="TLS"/> > <Engine name="Catalina" defaultHost="localhost"> > <Host name="localhost" appBase="webapps" > unpackWARs="true" autoDeploy="true"> > <Valve className="org.apache.catalina.valves.AccessLogValve" > directory="logs" > prefix="localhost_access_log" suffix=".txt" > pattern="%h %l %u %t "%r" %s %b" /> > </Host> > </Engine> > </Service> > </Server> > > Thanks, > Mason > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
