On Wed, Jul 22, 2020 at 07:38:52AM -0700, Greg Sims wrote:
> We have main.cf configured as follows:
>
> sender_dependent_default_transport_maps =
> randmap:{r192,r193,r194,r195,r196,r197,r198}
> smtp_connection_cache_on_demand=no
Can you share your reason for avoiding connection caching in general?
> Distributing the traffic across seven ip addresses is working well for
> all but one major transport.
What do you mean by "one major transport"? Do you mean "one major
destination"? Do you have a custom transport configured for that
destination?
> This one major transport is complaining about too many connections
> even with:
Transports don't complain about too many connections, but a set of
receiving servers might. Do these servers support lots of destination
domains?
One limitation of the way that Postfix handles destination concurrency
limits, is that these limits group messages by *recipient domain*, not
MX host.
A set of MX hosts that support many different domains are then
potentially handling concurrent connections for each domain, up to the
transport process limit, rather than the destination concurrency limit.
> majortransport_destination_concurrency_limit = 4
When you map a large provider's domain (say gmail.com) to a dedicated
transport, they might still see additional connections for all the
customer domains hosted by the same inbound MX hosts.
> I am configuring majortransport to use a single ip address in
> master.cf to reduce the number of connections.
Using but a single IP address increases the concurrent connects
per address. On the contrary, you'd want to spread the traffic
over multiple IPs, assuming that they don't aggregate connections
across netblocks large enough to encompass your entire IP pool.
> I hope this will allow an increased destination_concurrency_limit and
> better throughput for majortransport.
I don't see how it would help.
> This seems like a good scenario to use
> smtp_connection_cache_on_demand=yes.
This does not reduce connection concurrency, the main effect is to
reduce connection setup latency, and only if you don't use TLS,
otherwise you'd also want to enable use of tlsproxy(8) connection reuse.
So it may increase throughput, but at the cost of slightly higher
concurrency, since some of the connections may be idle at any give
time.
> Is it possible to configure this for just majortransport?
Yes, but you're unlikely to benefit from this if your goal
is reducing connection concurrency.
> It seems turning on the connection_cache for the entire installation
> will reduce the randomness and effectiveness of the randmap.
No, because the transport is selected first, and connection caching
never mixes transports. So once a message is assigned to a given
transport, it will only reuse connections cached for that transport.
Bottom line, you're struggling to think clearly about your problem,
and you're not quite clear on what the difference is between a
destination and transport, what destination concurrency means, ...
Read through more of the documentation:
TUNING_README
QSHAPE_README
qmgr(8)
Read the Nostarch press Postfix book by Patrick and Ralf, ...
If you need to avoid hitting concurrency limits at a large cloud
provider hosting many recipient domains, you may need to channel the
traffic to a dedicated Postfix instance that handles only traffic for
that provider, and has a sufficiently low *process* limit for the
smtp(8) transport.
That can be done by censoring the DNS replies for the IP addresses of
the MX hosts in question and replacing them with your own IPs that
direct traffic to the dedicated Postfix instance that handles traffic to
that provider.
Postfix would need a substantially modified queue manager to directly
limit traffic concurrency by destination IP address.
--
Viktor.
