Greetings,
my company sends about 170k emails every day, almost exclusively
online transactions confirmations. We are running Postfix 2.10.2 on two
RHEL 6.4 servers, and have been doing so for about 6 months. Prior to
that, we were running the RedHat-packaged 2.3.3 on RHEL 5.
For the past few weeks, we have had been experiencing slow delivery to
gmail.com, with the active queue slowly filling up, and delivery
sometimes taking up to an hour. This is definitely not a systematic
problem : we only had a couple hours with issues during the last week,
with the active queue going up to 400 messages on a single server,
whereas we're already reached 1500 messages in the active queue today.
Also, other domains remain unaffected.
The amount of messages being sent to gmail.com remains rather limited,
with barely 2000 mails an hour during peak times for that specific
domain, and our servers are almost flatlining CPU-wise. From what I see
in our logs, it seems that the gmail.com MX is occasionally a tad slow,
with a message transmission time of 4 or 5 seconds, and from them on the
next messages are stockpiling in my active queue, with the full delay
being spent in my own queue manager.
Our concurrency parameters should allow the backlog to be sent much
faster, which means I am most likely missing something obvious in my
configuration.
Here is a log extract illustrating the problem :
Jun 23 09:27:47 smtpp2 postfix_mtaprod/smtp[11690]: D98D62015A:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
delay=0.6, delays=0.02/0/0.04/0.54, dsn=2.0.0, status=sent (250 2.0.0 OK
1403508467 ek10si4003796wid.60 - gsmtp)
Jun 23 09:27:53 smtpp2 postfix_mtaprod/smtp[13218]: 90CEA2015A:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
delay=5.8, delays=0.01/0.78/0.2/4.8, dsn=2.0.0, status=sent (250 2.0.0
OK 1403508473 lk10si15946466wjc.177 - gsmtp)
Jun 23 09:27:56 smtpp2 postfix_mtaprod/smtp[13212]: 55AE8201D4:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
conn_use=2, delay=6.7, delays=0.02/5.4/0.02/1.2, dsn=2.0.0, status=sent
(250 2.0.0 OK 1403508476 lk10si15946466wjc.177 - gsmtp)
Jun 23 09:28:00 smtpp2 postfix_mtaprod/smtp[11688]: 1EA7820168:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
conn_use=3, delay=8.3, delays=0.02/5.4/0.02/2.9, dsn=2.0.0, status=sent
(250 2.0.0 OK 1403508480 lk10si15946466wjc.177 - gsmtp)
Jun 23 09:28:03 smtpp2 postfix_mtaprod/smtp[13218]: C219B201AA:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
conn_use=4, delay=10, delays=0.01/8.2/0.02/2.2, dsn=2.0.0, status=sent
(250 2.0.0 OK 1403508483 lk10si15946466wjc.177 - gsmtp)
Jun 23 09:28:07 smtpp2 postfix_mtaprod/smtp[13219]: 90877201E3:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
conn_use=5, delay=12, delays=0.02/8.8/0.03/3.4, dsn=2.0.0, status=sent
(250 2.0.0 OK 1403508487 lk10si15946466wjc.177 - gsmtp)
[...]
Jun 23 09:33:08 smtpp2 postfix_mtaprod/smtp[11687]: CD0E9201D4:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.27]:25,
conn_use=12, delay=104, delays=0.02/103/0.02/0.4, dsn=2.0.0, status=sent
(250 2.0.0 OK 1403508788 kq9si21959095wjc.136 - gsmtp)
[...]
Jun 23 10:20:00 smtpp2 postfix_mtaprod/smtp[20522]: C5E1E20223:
to=<x...@gmail.com>, relay=gmail-smtp-in.l.google.com[173.194.66.26]:25,
conn_use=23, delay=418, delays=0.02/417/0.02/0.4, dsn=2.0.0, status=sent
(250 2.0.0 OK 1403511600 u10si837587wjr.88 - gsmtp)
Here is our instance configuration, with a tiny bit of anonymizing. It's
pretty basic, no AV, no DKIM
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
command_directory = /usr/sbin
config_directory = /etc/
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
header_checks = regexp:/etc/header_checks
html_directory = no
inet_protocols = ipv4
local_header_rewrite_clients = static:all
mail_owner = mtaprod
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
masquerade_domains = XXX
mydomain = XXX
myhostname = XXX
mynetworks = XXX
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
queue_directory = /var/spool/postfix
readme_directory = no
sample_directory = /usr/share/doc/postfix-2.10.2/samples
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtp_destination_concurrency_limit = 10
smtp_destination_rate_delay = 1s
smtpd_recipient_restrictions = check_recipient_mx_access
regexp:/etc/recipient_mx_zero, permit_mynetworks,
reject_unauth_destination
syslog_name = postfix_mtaprod
transport_maps = hash:/etc/transport
unknown_local_recipient_reject_code = 550
What am I missing there? Any insight on this matter would be greatly
appreciated.
Thanks in advance,
Daniel
