On 03/15/2017 11:55 PM, Willy Tarreau wrote:
At least I can say I've seen many people enable it without understanding its impact, confusing it with tcp_tw_reuse, and copy-pasting it from random blogs and complaining about issues in production.
I currently wonder: What it the correct advise to an operator who needs to run one server instance that is meant to accept thousands of new, short-lived TCP connections per minute? The explanation text at https://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux seems to provide comprehensive advise, but its summary is, after all, somewhat disappointing:
The universal solution is to increase the number of possible quadruplets by using, for example, more server ports. This will allow you to not exhaust the possible connections with TIME-WAIT entries.
Assuming an operator has to deal with a given server executable, which does not provide a feature to "open many listening ports for the same purpose in parallel", this is hardly an option. (Of course, if you can start just N instead of 1 server instance, this becomes an option, but not everything is a simple, stateless web server.)
On the server side, do not enable net.ipv4.tcp_tw_recycle unless you are pretty sure you will never have NAT devices in the mix. Enabling net.ipv4.tcp_tw_reuse is useless for incoming connections.
So basically both options won't help the server operator.
On the client side, enabling net.ipv4.tcp_tw_reuse is another almost-safe solution. Enabling net.ipv4.tcp_tw_recycle in addition to net.ipv4.tcp_tw_reuse is mostly useless.
If you just operate the server, but not the (remote) clients, this is not relevant. Is the final verdict that unless a server software by itself offers to open N listen ports for the same purpose, there is no solution? Regards, Lutz Vieweg
