Hi,
We have a tomcat instance which is http2 enabled and it needs to serve
large number of requests using multiplexing, so we have configured our
instance as follows,
<Connector port="9191" URIEncoding="UTF-8"
sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
protocol="org.apache.coyote.http11.Http11Nio2Protocol"
maxThreads="50000" SSLEnabled="true"
compressibleMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml"
compression="on" minSpareThreads="25"
noCompressionUserAgents="gozilla, traviata" scheme="https" secure="true"
keystoreFile="conf/myfile.keystore" keystorePass="password"
socket.appReadBufSize="81920"
socket.appWriteBufSize="81920" socket.rxBufSize="251880"
socket.txBufSize="438000">
<UpgradeProtocol compression="on"
maxConcurrentStreamExecution="200"
maxConcurrentStreams="200"
className="org.apache.coyote.http2.Http2Protocol"/>
</Connector>
This instance mainly serves concurrent POST request which will have payload
of size, approx 1000-1500, which can be verified by tomcat logs
org.apache.coyote.http2.Http2Parser.validateFrame Connection [0], Stream
[19], Frame type [DATA], Flags [1], Payload size [*1195*]
We tested our server with the help of h2load as follows,
h2load -n100 -c1 -m100 https://localhost:9191/ -d '/agentRequest.txt'
We are getting this error as follows,
org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch Connection [0]
java.io.IOException: Unable to unwrap data, invalid status
[BUFFER_OVERFLOW]
at
org.apache.tomcat.util.net.SecureNio2Channel$2.completed(SecureNio2Channel.java:1041)
at
org.apache.tomcat.util.net.SecureNio2Channel$2.completed(SecureNio2Channel.java:1000)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127)
at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:158)
at
java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(UnixAsynchronousSocketChannelImpl.java:552)
at
java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:276)
at
java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:297)
at
org.apache.tomcat.util.net.SecureNio2Channel$2.completed(SecureNio2Channel.java:1027)
at
org.apache.tomcat.util.net.SecureNio2Channel$2.completed(SecureNio2Channel.java:1000)
at
org.apache.tomcat.util.net.SecureNio2Channel.read(SecureNio2Channel.java:1067)
at
org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper$VectoredIOCompletionHandler.completed(Nio2Endpoint.java:1153)
at
org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper.read(Nio2Endpoint.java:1026)
at
org.apache.tomcat.util.net.SocketWrapperBase.read(SocketWrapperBase.java:1012)
at
org.apache.coyote.http2.Http2AsyncParser.readFrame(Http2AsyncParser.java:61)
at
org.apache.coyote.http2.Http2Parser.readFrame(Http2Parser.java:69)
at
org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch(Http2UpgradeHandler.java:322)
at
org.apache.coyote.http2.Http2AsyncUpgradeHandler.upgradeDispatch(Http2AsyncUpgradeHandler.java:37)
at
org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:54)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:53)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at
org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1769)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
org.apache.tomcat.util.net.AbstractEndpoint.processSocket(AbstractEndpoint.java:1048)
at
org.apache.tomcat.util.net.SecureNio2Channel$HandshakeWriteCompletionHandler.completed(SecureNio2Channel.java:116)
at
org.apache.tomcat.util.net.SecureNio2Channel$HandshakeWriteCompletionHandler.completed(SecureNio2Channel.java:109)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127)
at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:158)
Why is this error is thrown? How can I configure tomcat to handle
concurrent POST requests which have a decent payload?
We have tried with various java clients like http-client-5-beta, jetty or
okhttp3 and spam requests to our tomcat using http2 multiplexing and we
found the time taken to process a requests increases(sometimes even 10x)
when load is increased.
We have tweaked all common configuration related to http2 on both client
and server side with no luck.
But same tomcat configuration can handle 10s of 1000s of get request
concurrently without a problem, its only creating problem with POST
requests.
What is wrong in our configuration?
Kindly someone shed some light.
Tomcat - 9.0.16
APR-1.2.18
OpenSSL-1.1.1a
JDK-10.0.2
OS - Ubuntu/Centos
HeapSize - 4GB
RAM -16GB
Kindly help
--
*With Regards,*
*Santhosh Kumar J*
