[pfx] TLS for SMTP Outbound -- Only One tlsproxy
We have been running without TLS for many years. Some of the ISPs are
beginning to complain about not sending mail using TLS. We enabled
outbound smtp as a result. Postfix receives email only from our
private network -- we do not use inbound smtpd_tls as a result.
Our main.cf contains:
smtpd_tls_cert_file =
smtpd_tls_key_file =
smtpd_tls_security_level = none
smtp_tls_security_level = may
I used smtp_tls_loglevel = 1 to verify that everything is working.
The ISPs are no longer complaining about the lack of TLS.
We have used connection caching in the past so we added:
* smtp_tls_connection_reuse=yes
* uncommented tlsproxy / maxproc = 0 in master.cf
* commented smtp_tls_loglevel -- no logging as a result
The outbound message transfer rate of our configuration is less than
500 emails per minute. I noted the following:
* we have four entries in master.cf for smtp -- each has a
unique ip address with maxproc=32; these are used with randmap{} in
main.cf
* queuing of inbound email is high
* the inbound email queue contains over 3,000 emails
* email average delay is over 400 seconds and 1,100
seconds for google.com where most of the email is delivered
* htop shows
* cpu utilization is low with Load average under 0.10 on
four physical processors and 4GB memory / 500MB used
* only one tlsproxy process running -- the highest cpu
utilization process most of the time
* 20 processes exist for each of our four ip address/smtp
entries in master.cf with maxproc=32
I am concerned that the queuing of inbound email is caused by there
only being one tlsproxy process. maxproc=0 seems to allow for an
unlimited number. We seem to have plenty of smtp processes as postfix
is not starting more of them to reach the maxproc=32.
Thank you for your assistance with this, Greg
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] TLS for SMTP Outbound -- Only One tlsproxy
TLS connections are being reused about 10% of the time for larger ISPs. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
Thank you Viktor.
Answers:
* smtp_connection_cache_on_demand = yes -- this was configured
Changes:
* certs back to defaults
* smtp_tls_loglevel = 1
Before enabling TLS our send rate was about 4K emails per minute -- we
are now seeing 300 to 500 per minute.
The email creation process is sending new mail via a private network.
We are in the middle of an ip address migration so -- email creation
is currently single threaded with 100 uSec delays between emails. I
can increase the concurrency/speed of the email creation process(es)
-- I fear it would only increase the size of the inbound email queue.
Here is a set of delays from the logs:
delays=0.01/2639/25/0.41
delays=0.01/2639/25/0.58
delays=0.01/2641/25/0.58
delays=0.01/2644/25/0.69
delays=0.01/2643/25/0.58
delays=0.01/2640/25/0.57
I scanned a large section of the logs | grep status=sent. These
delays are consistent throughout the peak demand period.
Best, Greg
On Tue, May 21, 2024 at 7:12 AM Viktor Dukhovni via Postfix-users
wrote:
>
> On Tue, May 21, 2024 at 06:51:08AM -0500, Greg Sims via Postfix-users wrote:
>
> > Our main.cf contains:
> > smtpd_tls_cert_file =
> > smtpd_tls_key_file =
> > smtpd_tls_security_level = none
>
> There's no point in configuring SMTP server certificates when TLS is
> disabled in the SMTP server. If the idea is to make tlsproxy(8)
> "happy", then try: "smtpd_tls_cert_file = none" and leave the key file
> setting at its default empty value.
>
> > smtp_tls_security_level = may
>
> Fine.
>
> > We have used connection caching in the past so we added:
> > * smtp_tls_connection_reuse=yes
>
> Presumably, you also have "smtp_connection_cache_on_demand = yes".
>
> > * uncommented tlsproxy / maxproc = 0 in master.cf
> > * commented smtp_tls_loglevel -- no logging as a result
>
> Level 1 is recommended when TLS is enabled. The additional logging
> overhead is quite modest.
>
> > The outbound message transfer rate of our configuration is less than
> > 500 emails per minute. I noted the following:
> >
> > * we have four entries in master.cf for smtp -- each has a
> > unique ip address with maxproc=32; these are used with randmap{} in
> > main.cf
>
> Fine.
>
> > * queuing of inbound email is high
> > * the inbound email queue contains over 3,000 emails
>
> So that's ~6 minutes of backlog... Plausibly due to downstream rate
> and/or concurrency limits
>
> > * email average delay is over 400 seconds and 1,100
> > seconds for google.com where most of the email is delivered
>
> If the backlog isn't growing, can you post the averages of the
> delays=a/b/c/d components?
>
> > * htop shows
> > * cpu utilization is low with Load average under 0.10 on
> > four physical processors and 4GB memory / 500MB used
>
> CPU was unlikely to be the problem.
>
> > * only one tlsproxy process running -- the highest cpu
> > utilization process most of the time
>
> Fine. More would be spawned if it got too busy to serve client requests
> in a timely manner.
>
> > * 20 processes exist for each of our four ip address/smtp
> > entries in master.cf with maxproc=32
> >
> > I am concerned that the queuing of inbound email is caused by there
> > only being one tlsproxy process.
>
> More plausibly the real issue is message delivery latency to the various
> destinations.
>
> > maxproc=0 seems to allow for an unlimited number. We seem to have
> > plenty of smtp processes as postfix is not starting more of them to
> > reach the maxproc=32.
>
> You could configure separate tlsproxy(8) services for each of the
> smtp(8) transports by overriding "tlsproxy_service_name" in master.cf,
> that will give you multiple tlsproxy(8) processes, but I guessing won't
> change much, if the issue is downstream delays.
>
> --
> Viktor.
> ___
> Postfix-users mailing list -- postfix-users@postfix.org
> To unsubscribe send an email to postfix-users-le...@postfix.org
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
TLS connection reuse is being used. About 10% of the connections are reused for large volume ISPs. Small volume ISPs do not see connection reuse. I believe this is as expected. I did some testing of our DNS setup. A DNS query using dig is less than 20 msec for both our primary and secondary dns servers in /etc/resolv.conf -- see below. Thanks, Greg [rsm@mail01 ~]$ cat /etc/resolv.conf # Generated by NetworkManager search raystedman.org nameserver 192.168.122.1 nameserver 8.8.4.4 [rsm@mail01 ~]$ dig @192.168.122.1 google.com ; <<>> DiG 9.11.36-RedHat-9.11.36-11.el8_9.1 <<>> @192.168.122.1 google.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45410 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 172 IN A 142.250.191.78 ;; Query time: 1 msec <<< ;; SERVER: 192.168.122.1#53(192.168.122.1) ;; WHEN: Tue May 21 15:56:06 PDT 2024 ;; MSG SIZE rcvd: 55 [rsm@mail01 ~]$ dig @8.8.4.4 google.com ; <<>> DiG 9.11.36-RedHat-9.11.36-11.el8_9.1 <<>> @8.8.4.4 google.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35863 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 300 IN A 142.250.191.46 ;; Query time: 16 msec ;; SERVER: 8.8.4.4#53(8.8.4.4) ;; WHEN: Tue May 21 15:57:47 PDT 2024 ;; MSG SIZE rcvd: 55 ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] TLS for SMTP Outbound -- Only One tlsproxy
TLS connection reuse is being used. About 10% of the connections are reused for large volume ISPs. Small volume ISPs do not see connection reuse. I believe this is as expected. I did some testing of our DNS setup. A DNS query using dig is less than 20 msec for both our primary and secondary dns servers in /etc/resolv.conf -- see below. Thanks, Greg [rsm@mail01 ~]$ cat /etc/resolv.conf # Generated by NetworkManager search raystedman.org nameserver 192.168.122.1 nameserver 8.8.4.4 [rsm@mail01 ~]$ dig @192.168.122.1 google.com ; <<>> DiG 9.11.36-RedHat-9.11.36-11.el8_9.1 <<>> @192.168.122.1 google.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45410 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 172 IN A 142.250.191.78 ;; Query time: 1 msec <<< ;; SERVER: 192.168.122.1#53(192.168.122.1) ;; WHEN: Tue May 21 15:56:06 PDT 2024 ;; MSG SIZE rcvd: 55 [rsm@mail01 ~]$ dig @8.8.4.4 google.com ; <<>> DiG 9.11.36-RedHat-9.11.36-11.el8_9.1 <<>> @8.8.4.4 google.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35863 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 300 IN A 142.250.191.46 ;; Query time: 16 msec ;; SERVER: 8.8.4.4#53(8.8.4.4) ;; WHEN: Tue May 21 15:57:47 PDT 2024 ;; MSG SIZE rcvd: 55 ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
Thank you again for your feedback on this issue. I watched the workload in real time this morning and now have more insight into what is happening. It appears the large ISPs are using TLS connection as a way to throttle incoming traffic. I looked at the inbound mail queue and found most of the traffic going to gmail.com. I believe this is because of the 20 & 25 seconds delays google.com is injecting into the TLS connection. I can also see no delays for the smaller ISPs with little traffic. Please see some data below. Best, Greg This is a sample of delays= for google.com -- 20 and 25 second delays: 0.01/11/20/0.73, 0.01/9.5/20/0.77, 0.01/0/25/0.74, 0.01/7.6/25/0.91, 0.01/6.9/25/1.1, 0.01/13/20/4.6, 0.01/14/25/0.56, 0.01/14/25/1.1, 0.01/0/0.22/0.72, 0.01/0/0.39/0.79, Here are delays= from yahoo -- 5.3 second delays: 0.01/0/10/4.1, 0.05/0/5.1/0.95, 0.01/0/5.2/0.68, 0.01/0/5.3/0.57, 0.01/0/0.45/0.42, 0.01/0/5.3/0.42, 0.01/0/5.3/0.34, 0.01/0.39/5.1/0.73, 0.01/0/1.2/2, 0.01/0/5.3/0.34, And from icloud -- little delay: 0.01/0/0.14/2.5, 0.01/0.02/0.15/2.7, 0.01/0/0.34/11, 0.01/0/0.13/4.9, 0.01/0/0.06/4.1, 0.01/0/0.13/1.5, ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
I have data collection homework to do -- and I will be happy to do it! Config data and "collate" is next after morning meetings. Here is some summary data by ISP from the logs: Email Ave Max Conn Relay SentDelayDelay Reused. google.com21,466 6681,5914,335 outlook.com 8,0568 2432,168 yahoodns.net 6,6178 381,135 prodigy.net 1,1267 420 icloud.com 6797 1980 comcast.net531 10 2240 This may supply some input on the amount of Connection Reuse we are seeing. I'm not sure why google.com is not reusing connections a higher percentage of time. We are always trying to send to gmail.com in the presence of the 25 second delay. Here is a 25 second instance by google with conn_use=2: [root@mail01 postfix]# journalctl -u postfix.service | grep 'May 22' | grep 604BE30A4ACA May 22 02:47:06 mail01.raystedman.org postfix/smtpd[45725]: 604BE30A4ACA: client=web01-p[192.168.122.11] May 22 02:47:06 mail01.raystedman.org postfix/cleanup[47234]: 604BE30A4ACA: message-id=. <8796a03d5ff2313c654830c3fb8cee03@swift.generated> May 22 02:47:06 mail01.raystedman.org postfix/qmgr[46605]: 604BE30A4ACA: from==gmail@devotion.raystedman.org>, size=34484, nrcpt=1 (queue active) May 22 03:13:22 mail01.raystedman.org t123/smtp[46725]: 604BE30A4ACA: to=<@gmail.com>, relay=gmail-smtp-in.l.google.com[142.251.2.26]:25, conn_use=2, delay=1576, delays=0.05/1550/25/0.84, dsn=2.0.0, status=sent (250 2.0.0 OK 1716372802 41be03b00d2f7-6578166a383si2117154a12.283 - gsmtp) May 22 03:13:22 mail01.raystedman.org postfix/qmgr[46605]: 604BE30A4ACA: removed Another 25 seconds by google without conn_use: [root@mail01 postfix]# journalctl -u postfix.service | grep 'May 22' | grep 4224D30C1817 May 22 02:47:14 mail01.raystedman.org postfix/smtpd[45721]: 4224D30C1817: client=web01-p[192.168.122.11] May 22 02:47:14 mail01.raystedman.org postfix/cleanup[47236]: 4224D30C1817: message-id=<3b852728e8ba8e7e4a0e83c4641dd5c5@swift.generated> May 22 02:47:14 mail01.raystedman.org postfix/qmgr[46605]: 4224D30C1817: from==gmail@devotion.raystedman.org>, size=34488, nrcpt=1 (queue active) May 22 03:12:57 mail01.raystedman.org t121/smtp[47173]: 4224D30C1817: to=<@gmail.com>, relay=gmail-smtp-in.l.google.com[142.250.141.27]:25, delay=1543, delays=0.01/1517/25/0.79, dsn=2.0.0, status=sent (250 2.0.0 OK 1716372777 41be03b00d2f7-66d4526311dsi4920983a12.891 - gsmtp) May 22 03:12:57 mail01.raystedman.org postfix/qmgr[46605]: 4224D30C1817: removed ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
I am having problems with "collate". I greped a 10 minute portion of our mail.log which created a 6.8M file. I ran "collate" on this file and collected the output -- a 796M file. I looked at the file and it seems to be filled with records like the following: May 22 02:10:00 mail01.raystedman.org postfix/smtpd[45725]: 122FC30C0DBA: client=web01-p[192.168.122.11] May 22 02:10:00 mail01.raystedman.org postfix/smtpd[45725]: 47ECB30C0DBD: client=web01-p[192.168.122.11] May 22 02:10:00 mail01.raystedman.org postfix/smtpd[45725]: 7BC8E30C178F: client=web01-p[192.168.122.11] I performed the following on the output file: [root@mail01 postfix]# cat 0522.02\:1x.log.collate | grep 122FC30C0DBA | wc 2816 19716 295764 This looks like 2,816 records of output for one email's smtpd record -- all with the same date/time stamp. I can see a few sequences like the following: May 22 02:10:00 mail01.raystedman.org postfix/qmgr[38428]: A01F430BFE03: from==att@devotion.raystedman.org>, size=34464, nrcpt=1 (queue active) May 22 02:10:03 mail01.raystedman.org postfix/qmgr[38428]: A01F430BFE03: removed Is this what you are looking for? Best, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
> It is assumed that you're not a victim of systemd-journald log mangling.
> It may be dropping some messages, and recording others out of order,
> breaking "collate". On Linux systems where systemd is doing the
> logging, you'll want to have Postfix writing its own log files directly,
> bypassing syslog. I have:
>
> main.cf:
> maillog_file = /var/log/postfix/log
>
> $ ls -ld /var/log/postfix{,/*}
> drwxrwxr-x. 2 root postdrop 47 May 22 23:40 /var/log/postfix
> -rw--- 1 root root 226 May 22 23:40 /var/log/postfix/log
> -rw--- 1 root postdrop 25747944 May 22 23:40
> /var/log/postfix/log.20240522-234048.gz
>
We are collecting the log files as input with:
journalctl -u postfix.service | grep 'May 22' >0522.log
It seems that this may be the issue with "collate". I will add
logging in main.cf.
We will be sending our daily devotion email in Chinese starting at
12:30 pacific time. I hope to have "collate" data soon afterwards.
Best, Greg
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
> > If the delay is with sending or receiving RSET, then the SMTP client > log "conversation with XXX timed out". I don't know if that has a > queue ID logged with that, though. Just grep for 'conversation with'. [root@mail01 postfix]# journalctl -u postfix.service | grep 'conversation with' returns null. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
> This is perhaps a good time to ask you for your full configuration,
> not just cherry-picked individual settings. Please post the outputs of:
>
> $ postconf -nf
> $ postconf -Mf
>
> with all whitespace (including linebreaks) preserved.
[root@mail01 postfix]# postconf -nf
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
command_directory = /usr/sbin
compatibility_level = 2
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd
$daemon_directory/$process_name $process_id & sleep 5
html_directory = no
inet_interfaces = all
inet_protocols = ipv4
mail_owner = postfix
maillog_file = /var/log/postfix/0522zh.log
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
maximal_backoff_time = 16m
meta_directory = /etc/postfix
minimal_backoff_time = 2m
mydestination =
mynetworks = 192.168.122.0/24, 127.0.0.0/8
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
queue_directory = /var/spool/postfix
queue_run_delay = 2m
readme_directory = /usr/share/doc/postfix/README_FILES
sample_directory = /usr/share/doc/postfix/samples
sender_dependent_default_transport_maps =
regexp:/etc/postfix/sender_relay.regexp, randmap:{t121,t122,t123,t124}
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
shlib_directory = /usr/lib64/postfix
smtp_connection_cache_on_demand = yes
smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt
smtp_tls_CApath = /etc/pki/tls/certs
smtp_tls_connection_reuse = yes
smtp_tls_loglevel = 1
smtp_tls_security_level = may
smtpd_tls_cert_file = /etc/pki/tls/certs/postfix.pem
smtpd_tls_key_file = /etc/pki/tls/private/postfix.key
smtpd_tls_security_level = none
unknown_local_recipient_reject_code = 550
[root@mail01 postfix]#
[root@mail01 postfix]#
[root@mail01 postfix]#
[root@mail01 postfix]# postconf -Mf
postlogunix-dgram n - n - 1 postlogd
smtp inet n - n - - smtpd
tlsproxy unix - - n - 0 tlsproxy
pickup unix n - n 60 1 pickup
cleanupunix n - n - 0 cleanup
qmgr unix n - n 300 1 qmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewriteunix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - n - - smtp
relay unix - - n - - smtp
-o syslog_name=postfix/$service_name
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discardunix - - n - - discard
local unix - n n - - local
virtualunix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache
rsmunix - - n - - smtp
-o smtp_bind_address=209.73.152.117
t121 unix - - n - 64 smtp
-o smtp_bind_address=209.73.152.121
-o smtp_helo_name=mail01-t121.raystedman.org
-o syslog_name=t121
t122 unix - - n - 64 smtp
-o smtp_bind_address=209.73.152.122
-o smtp_helo_name=mail01-t122.raystedman.org
-o syslog_name=t122
t123 unix - - n - 64 smtp
-o smtp_bind_address=209.73.152.123
-o smtp_helo_name=mail01-t123.raystedman.org
-o syslog_name=t123
t124 unix - - n - 64 smtp
-o smtp_bind_address=209.73.152.124
-o smtp_helo_name=mail01-t124.raystedman.org
-o syslog_name=t124
[root@mail01 postfix]#
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
> It is assumed that you're not a victim of systemd-journald log mangling.
> It may be dropping some messages, and recording others out of order,
> breaking "collate". On Linux systems where systemd is doing the
> logging, you'll want to have Postfix writing its own log files directly,
> bypassing syslog. I have:
>
> main.cf:
> maillog_file = /var/log/postfix/log
>
> $ ls -ld /var/log/postfix{,/*}
> drwxrwxr-x. 2 root postdrop 47 May 22 23:40 /var/log/postfix
> -rw--- 1 root root 226 May 22 23:40 /var/log/postfix/log
> -rw--- 1 root postdrop 25747944 May 22 23:40
> /var/log/postfix/log.20240522-234048.gz
>
> See https://www.postfix.org/MAILLOG_README.html and note that rotation
> does not (yet) delete old files, that's up to your cron job.
Postfix is now configured for logging at /var/log/postfix.
* added to main.cf:
maillog_file = /var/log/postfix/0522zh.log
* added to master.cf
postlog unix-dgram n - n - 1 postlogd
I then:
[root@mail01 0522zh]# perl ../collate.pl 0522zh.log >0522zh.log.collate
[root@mail01 0522zh]# ll
total 95M
-rw-r--r-- 1 root root 2.5M May 22 11:38 0522zh.log
-rw-r--r-- 1 root root 92M May 22 11:55 0522zh.log.collate
The output of collate does not appear to be useful -- it is the same
as when we are logging to memory.
I then ran:
[root@mail01 0522zh]# cat 0522zh.log.collate | grep -v
'192.168.122.11' >0522zh.log.collate.no_priv
[root@mail01 0522zh]# ll
total 96M
-rw-r--r-- 1 root root 2.5M May 22 11:38 0522zh.log
-rw-r--r-- 1 root root 92M May 22 11:55 0522zh.log.collate
# huge file size once again
-rw-r--r-- 1 root root 865K May 22 12:03 0522zh.log.collate.no_priv.
# reasonable file size after last grep
I scanned the last file. All I see is sequences like this:
May 22 11:30:03 mail01 postfix/smtpd[3177]: discarding EHLO keywords: CHUNKING
May 22 11:31:22 mail01 postfix/cleanup[3357]: 5592A30F1CB9:
message-id=<9d907dff3c240b43a5d92c8b41fcebfa@swift.generated>
May 22 11:31:22 mail01 postfix/qmgr[2563]: 5592A30F1CB9:
from==gmail@devotion.raystedman.org>,
size=45131, nrcpt=1 (queue active)
May 22 11:31:43 mail01 postfix/qmgr[2563]: 5592A30F1CB9: removed
May 22 11:30:03 mail01 postfix/smtpd[3177]: discarding EHLO keywords: CHUNKING
May 22 11:31:22 mail01 postfix/cleanup[3357]: 93D9D30F1E10:
message-id=<2e60eb7be562876e35be104949b788c3@swift.generated>
May 22 11:31:22 mail01 postfix/qmgr[2563]: 93D9D30F1E10:
from==gmail@devotion.raystedman.org>,
size=45151, nrcpt=1 (queue active)
May 22 11:31:43 mail01 postfix/qmgr[2563]: 93D9D30F1E10: removed
This is not the useful data you were looking for. I can not get this
to work. I would be glad to give you the log file offline.
Best, Greg
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
Thank you Viktor. All recommended changes have been made. I hope to collect useful "collate" data with our next distribution at Noon today pacific. I hope you have a great day! Greg > [root@mail01 postfix]# postconf -nf > > [root@mail01 postfix]# postconf -Mf ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: TLS for SMTP Outbound -- Only One tlsproxy
On Thu, May 23, 2024 at 7:07 AM Greg Sims wrote: > > Thank you Viktor. All recommended changes have been made. I hope to > collect useful "collate" data with our next distribution at Noon today > pacific. > Still having problems with the inbound smtpd from our private network flooding "collate". This could be caused by the process number always being the same. May 23 11:30:07 mail01 postfix/smtpd[12433]: E050630F1BE6: client=web01-p[192.168.122.11] May 23 11:30:08 mail01 postfix/smtpd[12433]: 2245330F1BEA: client=web01-p[192.168.122.11] May 23 11:30:08 mail01 postfix/smtpd[12433]: 5711630F1BEC: client=web01-p[192.168.122.11] root@mail01 0523zh]# ll total 99M -rw-r--r-- 1 root root 2.6M May 23 12:45 0523zh.log -rw-r--r-- 1 root root 94M May 23 12:46 0523zh.log.collate -rw-r--r-- 1 root root 1.9M May 23 12:51 0523zh.log.collate.no_priv I removed the smtpd records with grep -v to create the last file above with no private ip addresses (no_priv). One thing I noticed changing with the updates to main.cf and master.cf. I now see only "Untrusted TLS connection established". Last night (before the changes) we had 40K Trusted and TLS_Anonymous: 35, TLS_Untrusted: 78. Could this have something to do with deleting the default Certs associated with TLS? And now some "collate" data. The run starts at 11:30 -- we already have a long delay of 10 seconds: May 23 11:30:07 mail01 postfix/smtpd[12438]: discarding EHLO keywords: CHUNKING May 23 11:30:11 mail01 postfix/cleanup[12440]: F3B3630F1C5C: message-id=<62994d8ff3eb719d8ddc674c2cae6b38@swift.generated> May 23 11:30:11 mail01 postfix/qmgr[11016]: F3B3630F1C5C: from==hotmail@devotion.raystedman.org>, size=50829, nrcpt=1 (queue active) May 23 11:30:21 mail01 postfix/t124/smtp[12472]: Untrusted TLS connection established to hotmail-com.olc.protection.outlook.com[52.101.42.8]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (2048 bits) server-digest SHA256 May 23 11:30:21 mail01 postfix/t124/smtp[12472]: F3B3630F1C5C: to=<@hotmail.com>, relay=hotmail-com.olc.protection.outlook.com[52.101.42.8]:25, delay=11, delays=0.01/0.01/10/0.38, dsn=2.6.0, status=sent (250 2.6.0 <62994d8ff3eb719d8ddc674c2cae6b38@swift.generated> [InternalId=21569325775951, Hostname=CO3P220MB0782.NAMP220.PROD.OUTLOOK.COM] 60415 bytes in 0.104, 566.038 KB/sec Queued mail for delivery -> 250 2.1.5) May 23 11:30:21 mail01 postfix/qmgr[11016]: F3B3630F1C5C: removed Now 20 seconds: May 23 11:30:07 mail01 postfix/smtpd[12433]: discarding EHLO keywords: CHUNKING May 23 11:30:09 mail01 postfix/cleanup[12441]: 3634330F1BF4: message-id=<826855425363d60cc81d5b8f49e83579@swift.generated> May 23 11:30:09 mail01 postfix/qmgr[11016]: 3634330F1BF4: from==yahoo@devotion.raystedman.org>, size=50821, nrcpt=1 (queue active) May 23 11:30:29 mail01 postfix/t123/smtp[12455]: Untrusted TLS connection established to mta6.am0.yahoodns.net[98.136.96.75]:25: TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 May 23 11:30:30 mail01 postfix/t123/smtp[12455]: 3634330F1BF4: to=<@yahoo.com>, relay=mta6.am0.yahoodns.net[98.136.96.75]:25, delay=21, delays=0.01/0.01/20/0.54, dsn=2.0.0, status=sent (250 ok dirdel) May 23 11:30:30 mail01 postfix/qmgr[11016]: 3634330F1BF4: removed Towards the end of the run -- 30 seconds: May 23 11:30:07 mail01 postfix/smtpd[12433]: discarding EHLO keywords: CHUNKING May 23 11:33:49 mail01 postfix/cleanup[12690]: BFB3B30F279B: message-id=<6e7c80dbfbec093a18061cdca1ae4b9c@swift.generated> May 23 11:33:49 mail01 postfix/qmgr[11016]: BFB3B30F279B: from==gmail@devotion.raystedman.org>, size=50821, nrcpt=1 (queue active). May 23 11:35:28 mail01 postfix/t121/smtp[12668]: Untrusted TLS connection established to gmail-smtp-in.l.google.com[142.251.2.27]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 May 23 11:35:29 mail01 postfix/t121/smtp[12668]: BFB3B30F279B: to=<@gmail.com>, relay=gmail-smtp-in.l.google.com[142.251.2.27]:25, delay=100, delays=0.01/69/30/0.8, dsn=2.0.0, status=sent (250 2.0.0 OK 1716489329 d2e1a72fcca58-6f4d2b41c18si755622b3a.334 - gsmtp) May 23 11:35:29 mail01 postfix/qmgr[11016]: BFB3B30F279B: removed Max connections from hotmail: May 23 11:30:07 mail01 postfix/smtpd[12433]: discarding EHLO keywords: CHUNKING May 23 11:32:45 mail01 postfix/cleanup[12681]: 9433130F50E9: message-id=<6ace7587b9302b23a29b052a5503afb9@swift.generated> May 23 11:32:45 mail01 postfix/qmgr[11016]: 9433130F50E9: from==hotmail@devotion.raystedman.org>, size=50833, nrcpt=1 (queue active) May 23 11:32:45 mail01 postfix/t121/smtp[12559]: Untrusted TLS connection established to hotmail-com.olc.protection.outlook.com[52.101.11.16]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchang
[pfx] Capture Bounced Email Headers & Content
We found the following in our email log: May 26 00:35:57 mail01.raystedman.org postfix/t124/smtp[39065]: 0A7D630F1C7C: to== cecytebc.edu...@devotion.raystedman.org>, relay=aspmx.l.google.com[142.251.2.26]:25, delay=0.52, delays=0/0/0.21/0.31, dsn=5.7.26, status=bounced (host aspmx.l.google.com[142.251.2.26] said: 550-5.7.26 Unauthenticated email from raystedman.org is not accepted due to 550-5.7.26 domain's DMARC policy. Please contact the administrator of 550-5.7.26 raystedman.org domain if this was a legitimate mail. To learn about 550-5.7.26 the DMARC initiative, go to 550 5.7.26 https://support.google.com/mail/?p=DmarcRejection 98e67ed59e1d1-2bf5fe61ba7si4174351a91.147 - gsmtp (in reply to end of DATA command)) We delivered 1000s of email to Google in this time frame -- we received only five failures like the one above. I worked with the Google Team today and determined the following: (1) Our SPF DNS contains the IP Address this email was sent from. This implies DMARC should have passed -- as I believe DMARC requires failures for Both SPF and DKIM for a hard failure. (2) The sending IP Address is a valid relay for our domain at Google. The email should not have been Unauthenticated as a result. (3) I checked bou...@raystedman.org which in general ends up with all email that bounce and contains VERP -- this was not the case with this email. The Google Team would like to have the message headers of this email to further debug the issue. How can I use Postfix to capture the email headers/content in this case. Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
On Mon, May 27, 2024 at 3:40 AM Viktor Dukhovni via Postfix-users < postfix-users@postfix.org> wrote: > > You really should have posted "collate" output, which would have shown > the envelope sender address in the "qmgr active" log entry. Perhaps > the actual domain used did not have the expected SPF records. > I tried to collect "collate" data but the logs for this time period are no longer available. Thanks for the feedback Viktor, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
> On Mon, May 27, 2024 at 3:40 AM Viktor Dukhovni via Postfix-users < postfix-users@postfix.org> wrote: > You really should have posted "collate" output, which would have shown > the envelope sender address in the "qmgr active" log entry. Perhaps > the actual domain used did not have the expected SPF records. Wietse: notify_classes is working well. Postmaster is hosted by Google and we are seeing sequences like the following as a result. I had to move notify_classes to main.cf to cover all of our email. For debug it would likely be best to have Postmaster email remain on our server -- instead of sending it to Google. Please see below as I believe we may have a better understanding without the email headers. Viktor: The "collate" for this issue is enlightening. Here is an instance: May 28 02:11:41 mail01.raystedman.org postfix/bounce[19442]: B78BC305D5A9: postmaster non-delivery notification: 4A841305D5BE May 28 02:11:41 mail01.raystedman.org postfix/cleanup[19458]: 4A841305D5BE: message-id=<20240528091141.4a841305d...@mail01.raystedman.org> May 28 02:11:41 mail01.raystedman.org postfix/qmgr[16460]: 4A841305D5BE: from=, size=3187, nrcpt=1 (queue active) May 28 02:11:41 mail01.raystedman.org postfix/t124/smtp[19403]: Trusted TLS connection established to aspmx.l.google.com[142.250.141.27]:25: TLSv1.3 with cipher T LS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 May 28 02:11:41 mail01.raystedman.org postfix/t124/smtp[19403]: 4A841305D5BE: host aspmx.l.google.com[142.250.141.27] said: 421-4.7.26 Your email has been rate limited because it is unauthenticated. Gmail 421-4.7.26 requires all senders to authenticate with either SPF or DKIM. 421-4.7.26 421-4.7.26 Authentication results: 421-4.7.26 DKIM = did not pass 421-4.7.26 SPF [mail01.raystedman.org] with ip: [209.73.152.124] = did not pass 421-4.7.26 421-4.7.26 For instructions on setting up authentication, go to 421 4.7.26 https://support.google.com/mail/answer/81126#authentication d2e1a72fcca58-701bc33108esi1286635b3a.272 - gsmtp (in reply to end of DATA command) May 28 02:11:42 mail01.raystedman.org postfix/t124/smtp[19403]: Trusted TLS connection established to alt2.aspmx.l.google.com[74.125.126.27]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 May 28 02:11:42 mail01.raystedman.org postfix/t124/smtp[19403]: 4A841305D5BE: to=, orig_to=, relay= alt2.aspmx.l.google.com[74.125.126.27]:25, delay=1.2, delays=0/0/0.81/0.39, dsn=2.0.0, status=sent (250 2.0.0 OK 1716887502 ca18e2360f4ac-7eae2d6333asi30711039f.32 - gsmtp) May 28 02:11:42 mail01.raystedman.org postfix/qmgr[16460]: 4A841305D5BE: removed It appears that this bounce email was sent using the subdomain mail01.raystedman.org. There is no "from=<>" here but the error log seems to imply the SPF failure was associated with this subdomain. If this is the case, we need to add an SPF record for this subdomain. Please note we are already configured for raystedman.org and devotion.raystedman.org. Are there any other subdomains that Postfix would use? Please let me know and I will make the appropriate modifications to DNS. Thank you, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
I do see the "qmgr active" active with the from=<>. I added mail01.raystedman.org SPF to DNS as a result. Thanks again, Greg > ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
On Tue, May 28, 2024 at 6:49 AM Wietse Venema via Postfix-users < postfix-users@postfix.org> wrote: > In recent experience with my personal porcupine.org email address, > they not only want SPF or DKIM, they *also* want a DMARC policy > with p=quarantine or p=reject. We have run p=reject for years. DMARC is currently p=none because of the issue you are helping with. I feel like we have a solution now -- time will tell. I hope to be p=reject once again soon! Thanks Wietse, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Change Domain of "from="
Hello, We found the following in our email logs this morning. I ran "collate" and here is the result: May 29 02:10:04 mail01.raystedman.org postfix/bounce[31220]: AFC7030537E6: postmaster non-delivery notification: 7A80D32EDB2C May 29 02:10:04 mail01.raystedman.org postfix/cleanup[31245]: 7A80D32EDB2C: message-id=<20240529091004.7a80d32ed...@mail01.raystedman.org> May 29 02:10:04 mail01.raystedman.org postfix/qmgr[27525]: 7A80D32EDB2C: from=, size=3380, nrcpt=1 (queue active) May 29 02:10:04 mail01.raystedman.org postfix/t122/smtp[31017]: Trusted TLS connection established to aspmx.l.google.com[142.250.141.27]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/2 56 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 May 29 02:10:05 mail01.raystedman.org postfix/t122/smtp[31017]: 7A80D32EDB2C: host aspmx.l.google.com[142.250.141.27] said: 421-4.7.26 Your email has been rate limited because it is unauthenticated. Gmail 421-4.7.26 requires all senders to authenticate with either SPF or DKIM. 421-4.7.26 421-4.7.26 Authentication results: 421-4.7.26 DKIM = did not pass 421-4.7.26 SPF [mail01.raystedman.org] with ip: [209.73.152.122] = did not pass 421-4.7.26 421-4.7.26 For instructions on setting up authentication, go to 421 4.7.26 https://support.google.com/mail/answer/81126#authentication d2e1a72fcca58-6f8fc04d880si9913771b3a.16 - gsmtp (in reply to end of DATA command) May 29 02:10:05 mail01.raystedman.org postfix/t122/smtp[31017]: Trusted TLS connection established to alt2.aspmx.l.google.com[74.125.126.26]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 May 29 02:10:05 mail01.raystedman.org postfix/t122/smtp[31017]: 7A80D32EDB2C: to=, orig_to=, relay=alt2.aspmx.l.google.com[74.125.126.26]:25, delay=1.2, delays=0/0/0.92/0.3, dsn=4.7.26, status=deferred (host alt2.aspmx.l.google.com[74.125.126.26] said: 421-4.7.26 Your email has been rate limited because it is unauthenticated. Gmail 421-4.7.26 requires all senders to authenticate with either SPF or DKIM. 421-4.7.26 421-4.7.26 Authentication results: 421-4.7.26 DKIM = did not pass 421-4.7.26 SPF [mail01.raystedman.org] with ip: [209.73.152.122] = did not pass 421-4.7.26 421-4.7.26 For instructions on setting up authentication, go to 421 4.7.26 https://support.google.com/mail/answer/81126#authentication ca18e2360f4ac-7e9c1b21032si328213139f.60 - gsmtp (in reply to end of DATA command)) main.cf contains: # 24-05-28 # email comes from raystedman.org instead of mail0.raystedman.org # note: the mail01 subdomain does not need a SPF record in DNS as a result myorigin = raystedman.org I hoped this would allow the message being sent to be from=. Please note the qmgr record above shows the name of the sending machine -- mail01.raystedman.org. Thank you, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
>
>
> > main.cf contains:
> >
> > # 24-05-28
> > # email comes from raystedman.org instead of mail0.raystedman.org
> > # note: the mail01 subdomain does not need a SPF record in DNS as a
> result
> > myorigin = raystedman.org
> >
> > I hoped this would allow the message being sent to be
> > from=. Please note the qmgr record
> > above shows the name of the sending machine -- mail01.raystedman.org.
>
> How about using sender_canoical_maps?
>
> sender_canonical_maps = inline:{
> { double-bou...@mail01.raystedman.ora =
> double_bou...@raystedman.org } }
>
> Why are you sending these notifications to Google?
>
>
Hi Wietse,
Our design point of sending the bounces to a Gmail mailbox at Google may
not be the best -- but it is practical for us.
I wrote software that reviews the bounces in the Gmail mailbox and
unsubscribes email addresses from the daily devotion distribution as
needed.The software is very conservative in the way this is done.
Bounces 3 out of 5 consecutive days and only for certain types of errors.
If our software recognises the bounce, the bounce email is deleted by the
software from the mailbox. Once per month I log into Gmail and review the
remaining bounces manually. RSM uses Gmail for all of the people who work
& volunteer for the ministry. This pattern seemed to fit.
I am looking for a way to resolve our SPF issue. If sender_canonical_maps
is the solution, I will give it a try. Did you expect that using "myorigin
= raystedman.org" would resolve the SPF issue?
Thanks, Greg
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
On Wed, May 29, 2024 at 2:52 PM Wietse Venema via Postfix-users wrote: > Presumably you have to DKIM or SPF or DMARC for hostname.raystedman.org, > so any way to get double-bou...@raystedman.org should help. > > You have to be careful about mailer loops, though. > > Postfix gives special treatment to <> and > to avoid an infinite loop of notifications for failed notifications. Please note mail01 receives email from our private network Only. This email is created by our servers. mail01 does not receive email from the Internet. All of our MX records point to Google. If we can get the double-bounce to Google, there seems to be little chance of a mailer loop. We have two DMARC/DKIM/SPF setups: (1) email with domain raystedman.org is relayed through Google. This is our transactional email (subscription double opt-in and the like). (2) email with domain devotion.raystedman.org is sent directly onto the Internet. I am reluctant to create a third DMARC/DKIM/SPF for the double-bounce case which is now using domain mail01.raystedman.org. I created a SPF record for mail01.raystedman.org -- for tonight. This should be enough to get DMARC to pass when the double-bounce email is received by Google -- at least this is the hope. I will work on this again Thursday. Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
On Wed, May 29, 2024 at 5:49 PM Wietse Venema via Postfix-users wrote: > I think it's a bad idea to send your double bounces to a different site. > The Postfix design really wants to handle them locally. Thank you Wietse. I moved to a conservative configuration for tonight including deleting the SPF record I created for mail01.raystedman.org. We will likely see four or five SPF failures from Google which we know to be -- at least safe. I would like to capture the double-bounces on the local machine to get a look at the message headers. I'm sure this additional data will give me insight on the cause of the double-bounces which is now unclear -- at least to me. Perhaps you can give me an idea of how to capture just the double-bounces locally. Thanks again, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
On Thu, May 30, 2024 at 7:12 AM Wietse Venema via Postfix-users wrote: > > Greg Sims via Postfix-users: > > double-bounces which is now unclear -- at least to me. Perhaps you > > can give me an idea of how to capture just the double-bounces locally. > > 1) The postmaster recipient address for single or double bounces is > configurable. > https://www.postfix.org/postconf.5.html#notify_classes > https://www.postfix.org/postconf.5.html#bounce_notice_recipient > https://www.postfix.org/postconf.5.html#2bounce_notice_recipient > > 2) You can configure Postfix to deliver that address locally. > https://www.postfix.org/STANDARD_CONFIGURATION_README.html#some_local I believe I am ready to capture the double-bounce locally. This is main.cf: # 24-05-30 save the bounces locally at bounce-local notify_classes = 2bounce, bounce, resource, software bounce_notice_recipient = bounce-local 2bounce_notice_recipient = bounce-local virtual_alias_maps = hash:/etc/postfix/virtual This is virtual: bounce-localbounce-local@localhost and ran postmap /etc/postfix/virtual. Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
On Thu, May 30, 2024 at 12:27 PM Greg Sims wrote: > > I believe I am ready to capture the double-bounce locally. > > This is main.cf: > # 24-05-30 save the bounces locally at bounce-local > notify_classes = 2bounce, bounce, resource, software > bounce_notice_recipient = bounce-local > 2bounce_notice_recipient = bounce-local > virtual_alias_maps = hash:/etc/postfix/virtual > > This is virtual: > bounce-localbounce-local@localhost > > and ran postmap /etc/postfix/virtual. This does not look good -- May 30 11:30:40 mail01.raystedman.org postfix/t121/smtp[52641]: 4FE9D3061EF0: to=, orig_to=, relay=none, delay=0, delays=0/0/0/0, dsn=5.4.6, status=bounced (mail for localhost loops back to myself) May 30 11:30:40 mail01.raystedman.org postfix/bounce[52732]: warning: 4FE9D3061EF0: undeliverable postmaster notification discarded The only changes were to set up the local mailbox. Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
I set the following in main.cf mydestination = localhost and received the following in our logs: May 31 04:42:27 mail01.raystedman.org postfix/local[3978]: 7CE5C30F558E: to=, orig_to=, relay=local, delay=0, delays=0/0/0/0, dsn=5.1.1, status=bounced (unknown user: "bounce-local") May 31 04:42:27 mail01.raystedman.org postfix/bounce[3970]: warning: 7CE5C30F558E: undeliverable postmaster notification discarded It appears we are trying to deliver locally now -- a step in the right direction. bounce-local is not a rhel user of the local machine -- please note the entry we have in virtual. I updated main.cf as follows hoping this will help: mydestination = $myhostname, localhost.$mydomain, localhost We know that the SPF error for the double-bounce was trying to be sent to domain mail01.raystedman.org -- thus $myhostname. Thanks, Greg On Thu, May 30, 2024 at 2:52 PM Wietse Venema wrote: > > Greg Sims via Postfix-users: > > On Thu, May 30, 2024 at 12:27?PM Greg Sims wrote: > > > > > > I believe I am ready to capture the double-bounce locally. > > > > > > This is main.cf: > > > # 24-05-30 save the bounces locally at bounce-local > > > notify_classes = 2bounce, bounce, resource, software > > > bounce_notice_recipient = bounce-local > > > 2bounce_notice_recipient = bounce-local > > > virtual_alias_maps = hash:/etc/postfix/virtual > > > > > > This is virtual: > > > bounce-localbounce-local@localhost > > > > > > and ran postmap /etc/postfix/virtual. > > > > This does not look good -- > > > > May 30 11:30:40 mail01.raystedman.org postfix/t121/smtp[52641]: <<< NOTE > > SMTP > > 4FE9D3061EF0: to=, orig_to=, > > relay=none, delay=0, delays=0/0/0/0, dsn=5.4.6, status=bounced (mail > > for localhost loops back to myself) > > You are delivering mail for localhost with the Postfix SMTP CLIENT. > > Add localhost to main.cf:mydestination and do "postfix reload". > > Then, Postfix will use the local delivery agent for localhost mail. > > Wietse > > > > > May 30 11:30:40 mail01.raystedman.org postfix/bounce[52732]: warning: > > 4FE9D3061EF0: undeliverable postmaster notification discarded > > > > The only changes were to set up the local mailbox. > > > > Thanks, Greg > > ___ > > Postfix-users mailing list -- postfix-users@postfix.org > > To unsubscribe send an email to postfix-users-le...@postfix.org > > ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Change Domain of "from="
On Fri, May 31, 2024 at 8:01 AM Wietse Venema via Postfix-users wrote: > > Greg Sims via Postfix-users: > > I set the following in main.cf > > > > mydestination = localhost > > > > and received the following in our logs: > > > > May 31 04:42:27 mail01.raystedman.org postfix/local[3978]: > > 7CE5C30F558E: to=, orig_to=, > > relay=local, delay=0, delays=0/0/0/0, dsn=5.1.1, status=bounced > > (unknown user: "bounce-local") > > Well duh, where should she deliver the message? Yes, this was unfortunate. New rhel user created and verified at /var/spool/mail. Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
On Tue, May 28, 2024 at 8:12 AM Greg Sims wrote: > > On Tue, May 28, 2024 at 6:49 AM Wietse Venema via Postfix-users > wrote: > > > In recent experience with my personal porcupine.org email address, > > they not only want SPF or DKIM, they *also* want a DMARC policy > > with p=quarantine or p=reject. > > We have run p=reject for years. DMARC is currently p=none because of the > issue you are helping with. I feel like we have a solution now -- time will > tell. I hope to be p=reject once again soon! > > Thanks Wietse, Greg We have our bounce messages being stored in a local mailbox bounce-local -- this is working well. Unfortunately the SPF Failure we see in the logs is not being sent to bounce-local. Please see the following "collate" sequence: Jun 02 02:19:21 mail01.raystedman.org postfix/bounce[26402]: B9A1C305D596: sender non-delivery notification: EF978305D5BA Jun 02 02:19:21 mail01.raystedman.org postfix/cleanup[26400]: EF978305D5BA: message-id=<20240602091921.ef978305d...@mail01.raystedman.org> Jun 02 02:19:21 mail01.raystedman.org postfix/qmgr[1311]: EF978305D5BA: from=<>, size=36846, nrcpt=1 (queue active) Jun 02 02:19:22 mail01.raystedman.org postfix/t121/smtp[26247]: Trusted TLS connection established to aspmx.l.google.com[142.251.2.26]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 Jun 02 02:19:22 mail01.raystedman.org postfix/t121/smtp[26247]: EF978305D5BA: host aspmx.l.google.com[142.251.2.26] said: 421-4.7.26 Your email has been rate limited because it is unauthenticated. Gmail 421-4.7.26 requires all senders to authenticate with either SPF or DKIM. 421-4.7.26 421-4.7.26 Authentication results: 421-4.7.26 DKIM = did not pass 421-4.7.26 SPF [] with ip: [209.73.152.121] = did not pass 421-4.7.26 421-4.7.26 For instructions on setting up authentication, go to 421 4.7.26 https://support.google.com/mail/answer/81126#authentication d2e1a72fcca58-70242b097aasi4749745b3a.183 - gsmtp (in reply to end of DATA command) Jun 02 02:19:22 mail01.raystedman.org postfix/t121/smtp[26247]: Trusted TLS connection established to alt2.aspmx.l.google.com[74.125.126.26]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256 Jun 02 02:19:23 mail01.raystedman.org postfix/t121/smtp[26247]: EF978305D5BA: to==icloud@devotion.raystedman.org>, relay=alt2.aspmx.l.google.com[74.125.126.26]:25, delay=1.3, delays=0/0/0.89/0.41, dsn=2.0.0, status=sent (250 2.0.0 OK 1717319963 ca18e2360f4ac-7eafe6365f9si240806939f.105 - gsmtp) Jun 02 02:19:23 mail01.raystedman.org postfix/qmgr[1311]: EF978305D5BA: removed Two things caught my eye here: * Please note the message is being sent from=<> (qmgr). This is likely the cause of the SPF failure as there is no domain that can be used to lookup the SPF record. * The goal for the past period of time is to get a look at the headers of this message. Unfortunately the message is not being sent to bounce-local. No entry from process "local" above to send the message to the bounce-local user's mailbox. Here is the current main.cf setup: notify_classes = bounce, resource, software bounce_notice_recipient = bounce-local virtual_alias_maps = hash:/etc/postfix/virtual Would changing this to the following make any difference? notify_classes = 2bounce, bounce, resource, software bounce_notice_recipient = bounce-local 2bounce_notice_recipient = bounce-local virtual_alias_maps = hash:/etc/postfix/virtual We really need to see this message! Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
OK. I found the email in the bounce mailbox at the gmail level. The issue seems to be consistent with what we could see from the email logs only. The SPF fails because the email is being sent from domain mail01.raystedman.org. You tried (Wietse) for some time to control the "from domain" for this message. I am not aware that we found a solution. I believe it is time to add an SPF record for mail01.raystedman.org and see where this takes us. Thank you for breaking down this set of logs Wietse, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
We had another DMARC Failure last night. The email ended up at the gmail level. X-Original-Authentication-Results: mx.google.com; spf=none (google.com: mail01-t122.raystedman.org does not designate permitted sender hosts) smtp.helo=mail01-t122.raystedman.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=raystedman.org It appears that Google is looking for SPF information for one of the transports we use in randmap. Do we need to have SPF records in place for all of our transports? More message headers and "collate" of this incident are available on request. Thanks, Greg On Sun, Jun 2, 2024 at 7:02 PM Greg Sims wrote: > > OK. I found the email in the bounce mailbox at the gmail level. The > issue seems to be consistent with what we could see from the email > logs only. The SPF fails because the email is being sent from domain > mail01.raystedman.org. You tried (Wietse) for some time to control the > "from domain" for this message. I am not aware that we found a > solution. I believe it is time to add an SPF record for > mail01.raystedman.org and see where this takes us. > > Thank you for breaking down this set of logs Wietse, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Capture Bounced Email Headers & Content
Someone asked what was being sent. The email is being sent to a mailbox collector of bounces at the Gmail level. The email contains a VERP address of the original sender. We perform automated bounce processing for all email that make it to the bounce address at the Gmail level. These bounces come from our Postfix server and from many ISPs across the Internet. Please note that all of our MX Records point to Google. Our Postfix server is only accessible from a private network shared by our virtual machines -- it has no inbound exposure to the Internet. I choose to create an SPF entry in DNS for each of our Postfix Transports based on the feedback here. This is kind of awkward given the number of SPF records we have in our zone now -- but I believe it will work. Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Comcast Throttle
Hi There, We receive over 500 log entries per day from Comcast that look like this: Sep 18 03:05:07 mail0 r105/smtp[15929]: AE3378857BA: to=, relay=mx1.comcast.net[96.114.157.80]:25, delay=0.69, delays=0/0.01/0.6/0.08, dsn=4.1.0, status=deferred (host mx1.comcast.net[96.114.157.80] said: 421 4.1.0 75.126.209.105 Throttled - try again later. Please see http://postmaster.comcast.net/smtp-error-codes.php#RL01 (in reply to MAIL FROM command)) The postmaster url says (among other things): If the rate-limit is impacting your ability to deliver email, shortening your server's delivery retry interval may resolve the issue. We decreased the delivery retry interval. queue_run_delay = 3m minimal_backoff_time = 3m maximal_backoff_time = 30m We have also decreased the arrival rate of outbound email to Postfix for all of the email we send. We hoped that the number of deferrals would reduce over time -- but this has not been the case. This deferred / retry seems to take a good deal of resources from both Comcast and the sender of email. Is there anything we can do to reduce the impact? Thanks, Greg ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast Throttle
>> I have written postmas...@comcast.net and never received a reply. I have >> tried several times and will try again per your suggestion. >Probably better to go the web form route: >https://postmaster.comcast.net -> https://spa.xfinity.com/report The web form results in an automated email saying, "Your ip addresses are not blocked." from Comcast. This is a difficult group to interact with. On Sun, Sep 22, 2024 at 11:27 PM Viktor Dukhovni via Postfix-users < postfix-users@postfix.org> wrote: > On Sun, Sep 22, 2024 at 07:29:30PM -0500, Greg Sims wrote: > > > The only traffic we send on these ip addresses is a Bible > > Daily Devotion email using double opt-in subscriptions. > > Despite the double opt-in, some receivers will report your mail as spam, > either because they fail to distinguish between "Trash" and "Junk" when > choosing to delete a message, or because they no longer wish to receive > the messages, and choose to have them filtered, rather than > unsubscribe. > > > I have written postmas...@comcast.net and never received a reply. I > have > > tried several times and will try again per your suggestion. > > Probably better to go the web form route: > > https://postmaster.comcast.net -> https://spa.xfinity.com/report > > > I can see that the number of comcast deferrals reduces when I reduce the > > arrival rate of all our outbound email to postfix. One idea is to reduce > > the delivery rate of only comcast.net email using postfix. Is this > > suggested/possible? > > Yes, provided you can live with 1 message/sec or less (total under 86k > per day per transport). > > # > http://www.postfix.org/postconf.5.html#default_destination_rate_delay > transport_destination_rate_delay = 1 > > The actual parameter name depends on the name of the master.cf transport > to which you relegate comcast.net traffic. With a rate delay, the > destination concurrency is always equal to 1, (delivery spacing is > at least one second apart, so of course no concurrent deliveries). > > With rate delays, consider also: > > # > http://www.postfix.org/postconf.5.html#default_destination_concurrency_failed_cohort_limit > transport_destination_concurrency_failed_cohort_limit = 3 > > so that just a single delivery failure (failed or lost connection, > refused EHLO, ... ) does not throttle the transport. See also: > > http://www.postfix.org/TUNING_README.html#rope > http://www.postfix.org/SCHEDULER_README.html#concurrency_limitations > > These are qmgr(8) parameters, do not make the mistake of setting them in > master.cf as transport-specific overrides. > > -- > Viktor. > ___ > Postfix-users mailing list -- postfix-users@postfix.org > To unsubscribe send an email to postfix-users-le...@postfix.org > ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
