> > We have main.cf configured as follows:
> >
> > sender_dependent_default_transport_maps =
> > randmap:{r192,r193,r194,r195,r196,r197,r198}
> > smtp_connection_cache_on_demand=no
>
> Can you share your reason for avoiding connection caching in general?Our current mail server uses one ip address and it running so slow that we can not deliver daily email to our subscribers in a timely fashion. I found this strategy of using multiple ip addresses here: https://shami.blog/2016/04/randomize-source-ip-addresses-with-postfix/ The blog states that connection_cache=no will "Prevent the SMTP clients from caching connections, so sending multiple emails to a single domain does not end up using the same SMTP client." I interpreted this as connection_cache=no will increase the random effect of using this strategy. We are now seeing 30 times the throughput of our previous mail server. This is huge step in the right direction. One major destination (outlook.com) is complaining about the number of connections it is seeing using this strategy. This destination is running on a dedicated transport in master.cf that only specifies: outlook unix - - n - - smtp -o syslog_name=outlook transport.regexp has seven domains that use the "outlook:" transport. main.cf contains: outlook_initial_destination_concurrency = 4 outlook_destination_concurrency_limit = 4 outlook_destination_recipient_limit = 2 outlook_destination_rate_delay = 0s outlook_destination_concurrency_failed_cohort_limit = 10 outlook_destination_concurrency_positive_feedback = 1/3 outlook_destination_concurrency_negative_feedback = 1/8 What is the best way to configure for the following message from outlook.com in the maillog: said: 451 4.7.652 The mail server [<source_server_ip>] has exceeded the maximum number of connections. Please note the email is being created on a VM with <source_server_ip>. The email is then sent to our new mail server via SMTP. It appears outlook is complaining about the number of connections for email coming from the source_server_ip -- only one ip address. Possible solutions include: (1) reduce outlook_destination_concurrency_limit = 4 with a minimum of 2 (2) send emails from multiple ip addresses on source_server (lots of work!) (3) use only one IP on the mail server for outlook transport (I agree, likely no help) (4) other ideas? Thanks, Greg www.RayStedman.org On Wed, Jul 22, 2020 at 8:37 AM Viktor Dukhovni <postfix-us...@dukhovni.org> wrote: > On Wed, Jul 22, 2020 at 07:38:52AM -0700, Greg Sims wrote: > > > We have main.cf configured as follows: > > > > sender_dependent_default_transport_maps = > > randmap:{r192,r193,r194,r195,r196,r197,r198} > > smtp_connection_cache_on_demand=no > > Can you share your reason for avoiding connection caching in general? > > > Distributing the traffic across seven ip addresses is working well for > > all but one major transport. > > What do you mean by "one major transport"? Do you mean "one major > destination"? Do you have a custom transport configured for that > destination? > > > This one major transport is complaining about too many connections > > even with: > > Transports don't complain about too many connections, but a set of > receiving servers might. Do these servers support lots of destination > domains? > > One limitation of the way that Postfix handles destination concurrency > limits, is that these limits group messages by *recipient domain*, not > MX host. > > A set of MX hosts that support many different domains are then > potentially handling concurrent connections for each domain, up to the > transport process limit, rather than the destination concurrency limit. > > > majortransport_destination_concurrency_limit = 4 > > When you map a large provider's domain (say gmail.com) to a dedicated > transport, they might still see additional connections for all the > customer domains hosted by the same inbound MX hosts. > > > I am configuring majortransport to use a single ip address in > > master.cf to reduce the number of connections. > > Using but a single IP address increases the concurrent connects > per address. On the contrary, you'd want to spread the traffic > over multiple IPs, assuming that they don't aggregate connections > across netblocks large enough to encompass your entire IP pool. > > > I hope this will allow an increased destination_concurrency_limit and > > better throughput for majortransport. > > I don't see how it would help. > > > This seems like a good scenario to use > > smtp_connection_cache_on_demand=yes. > > This does not reduce connection concurrency, the main effect is to > reduce connection setup latency, and only if you don't use TLS, > otherwise you'd also want to enable use of tlsproxy(8) connection reuse. > So it may increase throughput, but at the cost of slightly higher > concurrency, since some of the connections may be idle at any give > time. > > > Is it possible to configure this for just majortransport? > > Yes, but you're unlikely to benefit from this if your goal > is reducing connection concurrency. > > > It seems turning on the connection_cache for the entire installation > > will reduce the randomness and effectiveness of the randmap. > > No, because the transport is selected first, and connection caching > never mixes transports. So once a message is assigned to a given > transport, it will only reuse connections cached for that transport. > > Bottom line, you're struggling to think clearly about your problem, > and you're not quite clear on what the difference is between a > destination and transport, what destination concurrency means, ... > > Read through more of the documentation: > > TUNING_README > QSHAPE_README > qmgr(8) > > Read the Nostarch press Postfix book by Patrick and Ralf, ... > > If you need to avoid hitting concurrency limits at a large cloud > provider hosting many recipient domains, you may need to channel the > traffic to a dedicated Postfix instance that handles only traffic for > that provider, and has a sufficiently low *process* limit for the > smtp(8) transport. > > That can be done by censoring the DNS replies for the IP addresses of > the MX hosts in question and replacing them with your own IPs that > direct traffic to the dedicated Postfix instance that handles traffic to > that provider. > > Postfix would need a substantially modified queue manager to directly > limit traffic concurrency by destination IP address. > > -- > Viktor. >
