Hello,
Currently our Postfix server is experiencing a problem with intermittent
SASL auth problems.
This mail server worked well up to a point, when the "fatal: no SASL
authentication mechanisms" errors started. From this point I started
researching and changing the mail server configuration with some
success, but unable to eradicate the problem completely.
The SASL errors appears when the mail queue has only 100 mails incoming
or when the bursts of 1000+ occur. Which has given me some gray hair
trying to troubleshoot.
Server performance is good, enough memory and the worst CPU usage I have
seen, still showed idle at 50% and no IOWAIT.
Dovecot logs, with debug enabled, do not show any errors or logs that
correlate with the fatal message in the mail log.
As our client base grows, this mail problem will simply become more
severe. So any assistance will be appreciated.
Main config changes made:
Increased default service counts for Dovecot and Postfix to the current
values shown in the configs below.
Added auth caching to Dovecot.
Changed the tls_random_reseed_period to stop a problem with an entropy
shortage for postfix during peak message times.
The server specs:
VM based: 8 CPU with 100% CPU share, 16GB Memory and SSD storage based.
The nature of the workload:
Business communications app hosted on Kubernetes, which sends out email
notifications for new messages, to users that have activated
notifications. This can be a steady stream of 1 to 5 messaged per second
for several ours and then a burst of hundreds to 1.5k messages in the
span of 1-2 minutes. The connection for these messages, from all client
instances, goes to postfix/submission.
The mail process in the app, is "dumb" it sends messages as fast as it
can and does not respond to mail server smtp response codes. Changes to
this behavior, are in development.
postfix/submission with smtpd -v output snippit, the entire log is 2GB:
Timeout after xsasl_dovecot_server_connect is shown.
Aug 1 13:16:22 localhost postfix/submission/smtpd[1575570]:
xsasl_dovecot_server_create: SASL service=smtp, realm=(null)
Aug 1 13:16:22 localhost postfix/submission/smtpd[1575570]: name_mask:
noanonymous
Aug 1 13:16:22 localhost postfix/submission/smtpd[1575570]:
xsasl_dovecot_server_connect: Connecting
Aug 1 13:16:32 localhost postfix/submission/smtpd[1575570]: fatal: no
SASL authentication mechanisms
Aug 1 13:16:33 localhost postfix/master[186920]: warning: process
/usr/lib/postfix/sbin/smtpd pid 1575570 exit status 1
Postconf -nf output:
address_verify_map = proxy:btree:$data_directory/verify_cache
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
broken_sasl_auth_clients = yes
content_filter = smtp-amavis:[127.0.0.1]:10024
default_process_limit = 300
disable_vrfy_command = yes
inet_interfaces = all
inet_protocols = ipv4
mailbox_size_limit = 0
maximal_queue_lifetime = 1d
message_size_limit = 31457280
milter_content_timeout = 30s
milter_default_action = accept
minimal_backoff_time = 1500s
mydestination = $myhostname
myhostname = mail.wilix.dev
mynetworks = 127.0.0.0/8 [::1]/128
myorigin = $myhostname
non_smtpd_milters = inet:127.0.0.1:12345
postscreen_access_list = permit_mynetworks
cidr:/etc/postfix/postscreen_spf_whitelist.cidr
postscreen_bare_newline_action = enforce
postscreen_bare_newline_enable = yes
postscreen_blacklist_action = enforce
postscreen_dnsbl_action = enforce
postscreen_dnsbl_sites = zen.spamhaus.org=127.0.0.[2..11]*3
bl.spameatingmonkey.net=127.0.0.2*2 bl.spamcop.net=127.0.0.2
dnsbl.sorbs.net=127.0.0.[2..15]
postscreen_dnsbl_threshold = 3
postscreen_greet_action = enforce
postscreen_greet_banner = Welcome, please wait...
postscreen_non_smtp_command_action = enforce
postscreen_non_smtp_command_enable = yes
postscreen_pipelining_action = enforce
postscreen_pipelining_enable = yes
proxy_read_maps = proxy:unix:passwd.byname
proxy:pgsql:/etc/postfix/sql-domains.cf
proxy:pgsql:/etc/postfix/sql-domain-aliases.cf
proxy:pgsql:/etc/postfix/sql-aliases.cf
proxy:pgsql:/etc/postfix/sql-relaydomains.cf
proxy:pgsql:/etc/postfix/sql-maintain.cf
proxy:pgsql:/etc/postfix/sql-relay-recipient-verification.cf
proxy:pgsql:/etc/postfix/sql-sender-login-map.cf
proxy:pgsql:/etc/postfix/sql-spliteddomains-transport.cf
proxy:pgsql:/etc/postfix/sql-transport.cf
proxy_write_maps = $smtp_sasl_auth_cache_name $lmtp_sasl_auth_cache_name
$address_verify_map
readme_directory = no
recipient_delimiter = +
relay_domains = proxy:pgsql:/etc/postfix/sql-relaydomains.cf
smtp_connect_timeout = 15s
smtp_helo_timeout = 60s
smtp_tls_CApath = /etc/ssl/certs
smtp_tls_connection_reuse = yes
smtp_tls_exclude_ciphers = EXPORT, LOW
smtp_tls_loglevel = 1
smtp_tls_security_level = may
smtpd_banner = $myhostname ESMTP
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_helo_required = yes
smtpd_milters = inet:127.0.0.1:12345
smtpd_recipient_restrictions = check_policy_service inet:127.0.0.1:9999
permit_mynetworks permit_sasl_authenticated check_recipient_access
proxy:pgsql:/etc/postfix/sql-maintain.cf
proxy:pgsql:/etc/postfix/sql-relay-recipient-verification.cf
reject_unverified_recipient reject_unauth_destination
reject_non_fqdn_sender
reject_non_fqdn_recipient reject_non_fqdn_helo_hostname
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_login_maps = proxy:pgsql:/etc/postfix/sql-sender-login-map.cf
smtpd_tls_CApath = /etc/ssl/certs
smtpd_tls_auth_only = no
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.wilix.dev/fullchain.pem
smtpd_tls_dh1024_param_file = ${config_directory}/dh2048.pem
smtpd_tls_eecdh_grade = strong
smtpd_tls_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, SRP, 3DES,
eNULL
smtpd_tls_key_file = /etc/letsencrypt/live/mail.wilix.dev/privkey.pem
smtpd_tls_loglevel = 1
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, SRP,
3DES, eNULL
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_protocols = !SSLv2, !SSLv3
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
smtpd_tls_session_cache_database =
btree:$data_directory/smtpd_tls_session_cache
smtpd_tls_session_cache_timeout = 86400s
smtpd_use_tls = yes
strict_rfc821_envelopes = yes
tls_random_reseed_period = 1800
transport_maps = proxy:pgsql:/etc/postfix/sql-transport.cf
proxy:pgsql:/etc/postfix/sql-spliteddomains-transport.cf
unknown_local_recipient_reject_code = 550
unverified_recipient_reject_code = 550
virtual_alias_domains = proxy:pgsql:/etc/postfix/sql-domain-aliases.cf
virtual_alias_maps = proxy:pgsql:/etc/postfix/sql-aliases.cf
hash:/etc/postfix/virtual
virtual_mailbox_domains = proxy:pgsql:/etc/postfix/sql-domains.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp
Postconf -Mf output:
smtp inet n - n - 1 postscreen
smtpd pass - - n - - smtpd
smtp-amavis unix - - n - 4 lmtp
-o lmtp_data_done_timeout=1200
-o lmtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
submission-amavis unix - - n - 4 lmtp
-o lmtp_data_done_timeout=1200
-o lmtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
dnsblog unix - - n - 0 dnsblog
tlsproxy unix - - n - 0 tlsproxy
submission inet n - n - 550 smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o tls_preempt_cipherlist=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_reject_unlisted_recipient=no
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o smtpd_helo_restrictions=
-o smtpd_client_connection_count_limit=150
-o smtpd_sender_restrictions=reject_sender_login_mismatch
-o milter_macro_daemon_name=ORIGINATING
-o content_filter=submission-amavis:[127.0.0.1]:10026
smtps inet n - y - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o
smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
-o content_filter=smtp-amavis:[127.0.0.1]:10026
pickup unix n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
qmgr unix n - n 300 1 qmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - n - - smtp
relay unix - - n - - smtp
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discard unix - - n - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache
maildrop unix - n n - - pipe flags=DRhu
user=vmail argv=/usr/bin/maildrop -d ${recipient}
uucp unix - n n - - pipe flags=Fqhu
user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail unix - n n - - pipe flags=F
user=ftn
argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp unix - n n - - pipe flags=Fq.
user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix - n n - 2 pipe flags=R
user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop}
${user} ${extension}
mailman unix - n n - - pipe flags=FR
user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop}
${user}
autoreply unix - n n - - pipe flags=
user=vmail:vmail argv=/srv/modoboa/env/bin/python
/srv/modoboa/instance/manage.py autoreply $sender $mailbox
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o smtpd_restriction_classes=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o
receive_override_options=no_header_body_checks,no_unknown_recipient_checks
-o local_header_rewrite_clients=
Doveconf -n output:
# 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.16 (09c29328)
# OS: Linux 5.15.0-113-generic x86_64 Ubuntu 22.04.3 LTS
# Hostname: localhost
auth_cache_size = 100 M
auth_cache_ttl = 4 hours
auth_master_user_separator = *
auth_mechanisms = plain login
auth_worker_max_count = 60
default_client_limit = 1600
default_process_limit = 400
default_vsz_limit = 512 M
dict {
quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
mail_location = maildir:~/Maildir
mail_plugins = quota
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
passdb {
args = /etc/dovecot/dovecot-sql-master.conf.ext
driver = sql
master = yes
pass = yes
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
quota = dict:User quota::proxy::quota
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = " imap lmtp sieve"
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-radicale {
group = radicale
mode = 0666
user = radicale
}
unix_listener auth-userdb {
user = vmail
}
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service imap {
executable = imap postlogin
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service pop3 {
executable = pop3 postlogin
}
service postlogin {
executable = script-login /usr/local/bin/postlogin.sh
user = modoboa
}
service stats {
unix_listener stats-reader {
group = vmail
mode = 0660
user = vmail
}
unix_listener stats-writer {
group = vmail
mode = 0660
user = vmail
}
}
ssl_cert = </etc/letsencrypt/live/mail.wilix.dev/fullchain.pem
ssl_cipher_list =
EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4
ssl_key = # hidden, use -P to show it
ssl_min_protocol = TLSv1
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = quota sieve
postmaster_address = postmas...@wilix.dev
}
protocol imap {
mail_max_userip_connections = 15
mail_plugins = quota imap_quota
}
protocol sieve {
mail_max_userip_connections = 15
}
_______________________________________________
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
