Thank you very much for your explanations. I'm going to test with smtpd_client_connection_count_limit = 1
Three days ago I added smtpd_client_connection_rate_limit = 10, wich limit the number of connection by a client to 10 by time unit; a time unit equal to 60s by default. I noticed that it works well and permit to slow down big mailers. As you write it, when a mailing list campain was in progress, I was able to see hundreds of mails arriving from a domain with tail -f /var/log/mail.log | grep cleanup tail -f /var/log/mail.log | grep 'postfix/cleanup.*@domain_of_big_mailer Yet, i'm going to test with smtpd_client_connection_count_limit = 1, wich looks like smtpd_client_connection_rate_limit and smtpd_client_message_(rate|count)_limit parameters. I will give you news in a about week . ----- Mail original ----- De: "Stan Hoeppner" <s...@hardwarefreak.com> À: "Jeremie CEINTREY" <jeremie.ceint...@pharmagest.com> Cc: postfix-users@postfix.org Envoyé: Samedi 31 Mars 2012 19:16:57 Objet: Re: performance problems On 3/30/2012 2:40 AM, Jeremie CEINTREY wrote: Due to lack of any evidence I'll have to make some coarse educated guesses here. > I often encounter performance problems with postfix and mailing-list. Mailing list servers. Implicates rapid and likely parallel delivery over a short interval. > I have a relay mail postfix filtering mail traffic for about 12 000 > mailboxes. Mailboxes are hosted on other mail servers. > > I use also amavisd-new with spammassassin and clamav. > Postfix version is 2.8 and i use it with postscreen configuration. This is likely not the problem. > My problem is that I often see that mailq grows up to 7000 mails and it takes > about 2 hours to deliver those mails. I don't understand why it is so slow. > When i take a look at sender with qshape -s active i see who are sending > mails and it is always mailling, but it is not spam traffic. Let's assume you don't have any obvious Postfix mis-configuration. So lets think this through logically, step by step. The first and obvious issue is that mail is coming in faster than it can go out which is why the queue is filling up. So we need to identify the cause of the slow outbound queue. A couple of possible causes: 1. Downstream servers are limiting your delivery rate 2. You storage system doesn't have enough IOPS performance to handle both the incoming write load *and* outgoing read load #1 should be identifiable by lots of premature disconnects and/or 4xx rejections from downstream servers. If this is indeed the problem you can have admins of those servers make necessary changes, or you can create individual relay transports configured with delays. #2 can be fixed by replacing the queue disk with a faster device with more IOPS capability, either a disk with higher RPM, an SSD, or multiple disks in a striped RAID configuration, accomplished either with a software RAID driver or with a hardware RAID controller. Both #1 and #2 can usually easily be fixed by limiting the rate of incoming mail, specifically by reducing the number of allowed parallel connections. Both mailing lists and bulk mailers tend to send a large volume of mail in a short period of time. The sending MTAs tend to do this by opening multiple connections to your MTA, Postfix in this case, which allows 50 connections per client by default. To limit the number of inbound parallel connections, you would change the value of the following parameter smtpd_client_connection_count_limit from the default of 50 to something much lower, such as 1. This will limit the number of connections from mailing list servers, bulk mailers, and spammers, while having no negative impact on normal inbound mail flow. Regular/normal mail delivery is usually accomplished with one email being sent over a single connection which is then closed upon successful delivery. With 50 open connections, a sending MTA such as a mailing list server, could potentially send hundreds of messages per second. If your disk isn't fast enough, which is likely your problem, it will spend all of its IOPS writing the inbound mail to the queue, which takes precedence over reads, causing read starvation and thus slow delivery. This is likely why your queue piles up and takes so long to drain. The net effect of this is that instead of your outbound queue piling up messages, these messages pile up in the outbound queue of the sending MTA, allowing you to receive them at a sane rate, and preventing your queue from piling up, and delaying delivery. An almost identical question came up on this list about a month or so ago. Setting smtpd_client_connection_count_limit=1 solved the OP's queue problem. It'll probably solve yours as well. Hope this information is helpful. Please let us know if this fixes your problem. -- Stan
