9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring) that will allow to configure limits on max "active" connections that can be allowed at any given time (https://github.com/apache/trafficserver/pull/6754) Given that this feature is based on tracking "active connections", it should work very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2 and stream multiplexing, this is no longer true and it isn't nearly as effective. One can still use an average number of concurrent streams/connection to configure the limits, but, ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3) would be to track and (rate) limit request level concurrency. There is some ongoing work on this and to align better with that work, we are proposing to make the below changes to existing configs with 9.x
1) Rename "proxy.config.net.max_connections_in" to "proxy.config.net.max_requests_in" 2) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_active_requests_in" 3) proxy.config.net.connections_throttle - This config will stay put to limit the max number of open connections (FD, rlimits etc) More context on the new changes - The idea is to tune active connections that can be handled (based on the available resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune. The primary goal is to limit the max active connections allowed at any given instant. The resource requirement for an idle/inactive vs active connections are completely different - For e.g an idle connection really only consumes memory resource, while an active connection consumes CPU, network besides memory. And allowing to tune/cap the max active connections based on the deployed capacity for the resources available, would make timeout tuning almost redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which is very hard as it's hard to justify how large or small we want the active timeout to be or keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much higher than the default, while during peak hour, we'd want to limit it to ensure we are not starving resources on one connection. Note: there's one little TODO item here. PluginVC's connections are not tracked in the NetVC's active queue because these are bogus internal connections. However, for some users, internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume plenty of resources. Since these internal requests don't impact ingress network, and have a slightly different resource requirements than client originated requests, it might be better to track these using a separate config for internal requests. Will follow that part up with a separate PR.
