Paul,
On 5/18/21 07:44, Paul P Wolf wrote:
Hi,
I am trying to run a spring boot application with an embedded tomcat. In a
scenario, where there is a lot of load on the service, I would like tomcat to
refuse connections or return something like a 503 Service Unavailable. From
what I understand, you could have this behaviour by setting maxConnections and
and any additional connections get refused. At least this was how the old
blocking io acceptor worked, from what I understand.
The documentation says "The maximum number of connections that the server will
accept and process at any given time. When this number has been reached, the server will
accept, but not process, one further connection. This additional connection be blocked
until [...]".
However the documentation doesn't really state what happens if maxConnection+2
connections are reached.
I tried to run my application with following settings (embedded-tomcat 9.0.45):
maxConnections=3
acceptCount=2
maxThreads=1
processorCache=1
I created an endpoint, which just sleeps for a few seconds and
returns a string. When I create 50 separate connection via curl
instances to call that service I see the following behaviour with the
NIO Acceptor:
* 6 http connections are accepted immediately (maxThreads +
acceptCount + maxConnections)
The maxThreads setting should not be relevant, here. maxConnections
counts the total connections without regard to how many of them are
actually having work done (which requires a thread from maxThreads).
So accepting 6 connections really means:
maxConnections (3) + acceptCount (2) + 1 (because maxConnections says it
will accept 1+maxConnections, which is a little confusing IMO).
* 44 http connections aren't established just, but neither are they
refused. I will call them "blocked", but different from the
specification those are 44 blocked connections and not just 1
What is the TCP state of the first 6 connections? What about the other
44? What is the difference between "accepted" and "blocked", and how are
you telling them apart?
* once the first request finishes, the latest (blocked) requests
gets a connection and is being processed (not a request from the
accept-queue or one of the other already established connections)
So the queue is behaving "unfairly"?
* when there are no further blocked requests, the requests still
get processed in last in first out order
More unfairness.
You didn't post your <Connector> configuration (which is pretty critical
for trying to think about these things), but I suspect you aren't using
an <Executor>, which may ensure fairness. (Older Tomcats used an
internal thread pool which was NOT an "Executor" but later Tomcat should
always be using an Executor, which was intended to enforce fairness. Hmm.)
* I see some timeouts after a while with this setup, depending on
what timeouts I set on curl. The requests without an established
connection timeout with "connect-timeout" parameter and the ones with
established connections depending on the "max-time" parameter.
When you get a timeout, what kind of timeout are you encountering (on
the client side)? Is this a "connect timeout" or a "read timeout"?
Now I have a lot of questions and I suspect some bugs, but I wanted to ask on
this list first:
1. Is there a way to refuse connections, instead of blocking them?
maxConnections + acceptCount *should* be doing this. Remember that
acceptCount is just a suggestion to the OS. The OS is free to always
accept 65535 connections into its TCP/IP stack whether you say so or
not. Likewise, if you are connecting through other network devices (e.g.
load-balancer, whatever), there may be different rules in different
places. I'm assuming you are connecting directly to Tomcat on localhost,
but you didn't specify.
2. I can't control the connect-timeout of the clients - is there a
way to control it on the server side? (I tried to change
keepAliveTimeout and connectionTimeout without success)
It's impossible for the server to instruct the client to time-out. The
situation here is that the client can't reach the server therefore the
server can't communicate anything to the client.
If you can't change the connect-timeout of the clients, do you know what
the effective timeout actually is?
3. Why doesn't the specification state what happens if many more
requests are being made than maxConnections allows for? (esp. with
the behaviour being different among acceptors)
Which specification? The Tomcat documentation explains what happens with
0-maxConnections, then maxConnections + 1, then maxConnection + N where
N > 1, so ... what are you actually asking?
4. The processing order is troublesome - early requests get starved
by following ones. Why is this lifo instead of fifo?
Please try explicitly configuring <Executor> and see if that changes
anything.
5. The number of accepted connections is unintuitively. Why are
active connections with requests either being processed or being in
the accept queue not counted as active connections (actual
connections = maxConnections + acceptCount + maxThreads)?
Tomcat cannot see the OS's TCP backlog (the "accept queue") so nothing
can be counted there. The only thing Tomcat can see is up to
maxConnections + 1 at which point the acceptor thread will stop
accepting incoming connections (and those client connections will
queue-up in the TCP stack's backlog).
6. What is the difference between requests in the accept queue and
requests outside the queue, but still within the maxConnections
limit?
Let's be clear about nomenclature. When you say "accept queue", I think
"TCP stack backlog" which is 100% in the OS and not visible to Tomcat.
Tomcat itself has its own queue of incoming connections (ones that have
been accept()ed but no thread has yet been assigned). I suppose you mean
this Tomcat-queue when you say "requests outside the queue, but still
within the maxConnections limit"?
The difference is that the former (backlog) is not visible to Tomcat at
all (or any server that you might be using, this is not Tomcat- or
Java-specific) and the latter (having been accept()ed by Tomcat) are
being managed by Tomcat.
Any insight would be appreciated. If there is anything in here, that
I should report as bug, please let me know. I am happy to file one.
We will need more information. In your next post, please provide either
your <Connector> if you are using XML-based configuration or all your
code that is creating/configuring the Tomcat Connector. Also, please
give some default about your JVM, OS, and kernel version, etc.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
