reject_unknown_client_hostname and 450s
When reject_unknown_client_hostname triggers on an NXDOMAIN it returns a 550 error, which is great. When it triggers because there is no PTR record, it returns a 450 error, which is also great… except. What I see is servers that connect hundreds of times, getting 450 errors and ignoring them and trying to send their spam again and again and again. I have some IPs that have tried to connect hundreds of times to send a message that is always going to generate a 450 error since the host does not have a PTR record and never will. I have over 10,000 of these failures on an average day. Does anyone have any suggestions? I am thinking about writing a fail2ban action for them that triggers after 5 or 10 attempts with a long ban, but I am not sure that's a good idea. Or should I just stop worrying and figure the amount of resources being used is insignificant? -- sometimes ascii is the best use of bandwidth... Tonya Engst
Re: reject_unknown_client_hostname and 450s
On 2013-06-30 LuKreme wrote: > When reject_unknown_client_hostname triggers on an NXDOMAIN it returns > a 550 error, which is great. When it triggers because there is no PTR > record, it returns a 450 error, which is also great… except. > > What I see is servers that connect hundreds of times, getting 450 > errors and ignoring them and trying to send their spam again and again > and again. > > I have some IPs that have tried to connect hundreds of times to send a > message that is always going to generate a 450 error since the host > does not have a PTR record and never will. I have over 10,000 of these > failures on an average day. > > Does anyone have any suggestions? I am thinking about writing a > fail2ban action for them that triggers after 5 or 10 attempts with a > long ban, but I am not sure that's a good idea. > > Or should I just stop worrying and figure the amount of resources > being used is insignificant? I'd say fail2ban is the way to go about this. If you want to be on the safe side, make the threshold somewhat higher and extend the lockout period. Regards Ansgar Wiechers -- "Abstractions save us time working, but they don't save us time learning." --Joel Spolsky
smtpd_banner incorrect for 2nd domain
I am in the process of setting up a second domain for mail, but am at a bit of a roadblock. When I check things with mxtoolbox smtp test, everything is correct on my primary domain, but on the second one I get a message "Warning - Reverse DNS does not match SMTP Banner". Reverse DNS is correct (I have seperate IPs for each domain), so the smtpd_banner is sending the wrong domain (mydomain1 instead of mydomain2) So I understand what the problem is, (I think) I just don't know how to fix it. I have looked at the virtual domain hosting howto, but am unable to make sense of most of it. And google searched aren't helping me much either. Any assistance is greatly appreciated. Also, below are my postconf -n and master.cf outputs with domain names removed for reference. I am running all this on Debain 7 if it matters. Thanks! postconf -n: alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no broken_sasl_auth_clients = yes config_directory = /etc/postfix home_mailbox = mail/ inet_interfaces = all mailbox_command = /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT" mailbox_size_limit = 0 mydestination = mydomain1.org, hostname, hostname.mydomain1.org, mail.mydomain1.org, localhost.localdomain, localhost myhostname = mydomain1.org mynetworks = 127.0.0.0/8 [:::127.0.0.0]/104 [::1]/128, 10.8.0.0/24, 65.191.6.16 myorigin = /etc/mailname readme_directory = no recipient_delimiter = + smtp_tls_note_starttls_offer = yes smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_use_tls = yes smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = $myhostname smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_tls_cert_file = /etc/ssl/certs/postfix.pem smtpd_tls_key_file = /etc/ssl/private/postfix.key smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtpd_use_tls = yes virtual_alias_domains = mydomain2.com virtual_alias_maps = hash:/etc/postfix/virtual master.cf: smtp inet n - - - - smtpd -o content_filter=spamassassin submission inet n - - - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING smtps inet n - - - - smtpd -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING #628 inet n - - - - qmqpd pickupfifo n - - 60 1 pickup cleanup unix n - - - 0 cleanup qmgr fifo n - n 300 1 qmgr #qmgr fifo n - - 300 1 oqmgr tlsmgrunix - - - 1000? 1 tlsmgr rewrite unix - - - - - trivial-rewrite bounceunix - - - - 0 bounce defer unix - - - - 0 bounce trace unix - - - - 0 bounce verifyunix - - - - 1 verify flush unix n - - 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - - - - smtp # When relaying mail as backup MX, disable fallback_relay to avoid MX loops relay unix - - - - - smtp -o smtp_fallback_relay= # -o smtp_helo_timeout=5 -o smtp_connect_timeout=5 showq unix n - - - - showq error unix - - - - - error retry unix - - - - - error discard unix - - - - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - - - - lmtp anvil unix - - - - 1 anvil scacheunix - - - - 1 scache # # # Interfaces to non-Postfix software. Be sure to examine the manual # pages of the non-Postfix software to find out what options it wants. # # Many of the following services use the Postfix pipe(8) delivery # agent. See t
Re: reject_unknown_client_hostname and 450s
LuKreme: > When reject_unknown_client_hostname triggers on an NXDOMAIN it > returns a 550 error, which is great. When it triggers because there > is no PTR record, it returns a 450 error, which is also great? > except. That is incorrect. The 450 code is for errors where lookup failed (no result instead of "does not exist"). Wietse
Re: smtpd_banner incorrect for 2nd domain
Cliff Dunn: > I am in the process of setting up a second domain for mail, but am at a > bit of a roadblock. When I check things with mxtoolbox smtp test, > everything is correct on my primary domain, but on the second one I get > a message "Warning - Reverse DNS does not match SMTP Banner". Reverse > DNS is correct (I have seperate IPs for each domain), so the I think you missed the BASIC_CONFIGURATION_README document which among others covers the Postfix server hostname and IP address. One Postfix server can host lots of domains on the same IP address. There is no need to have a different IP address and SMTP server banner for each domain. If you must, you can operate separate Postfix servers for each domain. In that case each Postfix server must have its own main.cf inet_interfaces and main.cf myhostname settings. See MULTI_INSTANCE_README.html. Wietse
RE: cert error on outlook when send email using ssl
From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Jeroen Geilman Sent: 29 June 2013 22:42 To: postfix-users@postfix.org Subject: Re: cert error on outlook when send email using ssl On 06/29/2013 08:25 PM, kazabe wrote: > Hi. > > Im trying to use postfix with ssl. Now is working, but i have a > little situation with the outloook clients. > > always to send a email, see a message > > "The name of the security certificate is invalid or does not match the name > of the site" Well, is it invalid ? Does it match the name of the site ? These things matter, for TLS. (You should not be using SMTPS) > The message is sended after accept the message, but the end users are affraid > with this message. So tell them not to be afraid! There are only a few things you can do to "fix" this situation: 1. provide a valid and trusted certificate (this will cost either effort or money), or 2. accept the way things are. > Im looking o google about to how to solve, but all the info are related with > ms exchange and i use postfix. > Can you share me some clues to solve it? X.509 certficates are normally checked for 3 properties: 1. is it valid (i.e. does the current date lie between the valid-from and valid-to attributes of the certificate)? 2. does the CN (common name) attribute of the certificate correspond to the name of the server you're connecting to ? 3. is the issuer of this certificate trusted by the client ? The first two are trivially corrected by you. The last one requires either that you get clients to trust your CA, or that you buy a certificate from a CA who is already trusted. -- J. --- StartSSL will do you a free certificate. https://www.startssl.com/ Bart...
Re: reject_unknown_client_hostname and 450s
On 6/30/2013 3:12 AM, LuKreme wrote: > When reject_unknown_client_hostname triggers on an NXDOMAIN it returns a 550 > error, which is great. When it triggers because there is no PTR record, it > returns a 450 error, which is also great… except. What you're seeing is the PTR lookup fails with a temporary DNS lookup error, which always results in a 450 deferral. > > What I see is servers that connect hundreds of times, getting 450 errors and > ignoring them and trying to send their spam again and again and again. > > I have some IPs that have tried to connect hundreds of times to send a > message that is always going to generate a 450 error since the host does not > have a PTR record and never will. I have over 10,000 of these failures on an > average day. > > Does anyone have any suggestions? I am thinking about writing a fail2ban > action for them that triggers after 5 or 10 attempts with a long ban, but I > am not sure that's a good idea. > > Or should I just stop worrying and figure the amount of resources being used > is insignificant? Just ignore them is usually the best action. but if their DNS is slow to fail and they make lots of parallel connections, they can tie up all your smtpd processes. If that happens, fail2ban is a good solution. -- Noel Jones
Re: cert error on outlook when send email using ssl
On Sun, Jun 30, 2013 at 5:33 AM, Bart J. Smit wrote: > --- > StartSSL will do you a free certificate. https://www.startssl.com/ +1 to Bart's comment. Just get a free cert from StartCom. I have no affiliation, and YMMV, but back in 2011 I wrote a howto for genning a free cert there and plugging it in to Postfix: http://stevejenkins.com/blog/2011/09/how-to-use-a-free-startssl-certificate-in-postfix-for-ssltls/ SteveJ
Re: cert error on outlook when send email using ssl
Jerry schreef op 2013-06-29 22:05: On Sat, 29 Jun 2013 13:25:50 -0500 kazabe articulated: Hi. Im trying to use postfix with ssl. Now is working, but i have a little situation with the outloook clients. always to send a email, see a message "The name of the security certificate is invalid or does not match the name of the site" The message is sended after accept the message, but the end users are affraid with this message. Im looking o google about to how to solve, but all the info are related with ms exchange and i use postfix. Can you share me some clues to solve it? Why not just get a valid certificate? Some valid certificates require an intermediate certificate to be installed and presented together with signed certificate, but many forget to install it or do it incorrectly. Hans
Re: reject_unknown_client_hostname and 450s
On 6/30/2013 3:12 AM, LuKreme wrote: > When reject_unknown_client_hostname triggers on an NXDOMAIN it returns a 550 > error, which is great. When it triggers because there is no PTR record, it > returns a 450 error, which is also great… except. > > What I see is servers that connect hundreds of times, getting 450 errors and > ignoring them and trying to send their spam again and again and again. > > I have some IPs that have tried to connect hundreds of times to send a > message that is always going to generate a 450 error since the host does not > have a PTR record and never will. I have over 10,000 of these failures on an > average day. > > Does anyone have any suggestions? Hosts that have no PTR/rDNS are almost certainly end user broadband PCs. Which means the clients are likely spambots. They ignore rejections, and they do not retry. They simply keep pumping out new connections. If they're all currently being rejected, and are not tying up your smtpds, then as Noel suggested, simply ignore it. If single clients are using concurrent connections and eating too many smtpds then fail2ban is one option. Postscreen is another. Or... Postfix allows 50 concurrent connections per client by default with a max of 100 smtpds. Set smtpd_client_connection_count_limit to something like 10 and watch your log daily for a week or so to make sure you're not burdening legit clients. The proper value here, if any, depends on your mail flow. This will limit concurrent connections of all clients. -- Stan
Re: PATCH: Option to log clients that execute invalid commands or disconnect with no email delivery
On 28/06/13 22:30, Wietse Venema wrote: > Wietse Venema: >> John Fawcett: >>> I use fail2ban in order to block some types of apparently malicious >>> connections to postfix when the clients keep retrying. For example the >> As you agree logging every failed command would not be safe by >> default. >> >> On the other hand, logging the command name (even without) parameters >> for every [45]XX response could be tricky. Adding IF statements all >> over the code is undesirable, so this would require a structural >> change to the command reader and responder. >> >> What about a one-line change, such that the SMTP server logs the >> existing per-session error counter when the connection is closed? >> >> This counter is reset upon successful completion of a (MAIL, RCPT, >> DATA, end-of-data) sequence. This should be sufficient to expose >> clients that hammer your server with unimplemented AUTH commands. > Example: > > Jun 28 16:27:25 spike postfix/smtpd[65532]: disconnect from > camomile.cloud9.net[2604:8d00:0:1::3] error_count 0 > > As per the patch below for any Postfix version ever released. > > Wietse > > *** ./src/smtpd/smtpd.c- Sun Jun 23 11:10:02 2013 > --- ./src/smtpd/smtpd.c Fri Jun 28 16:26:41 2013 > *** > *** 4989,4995 >* After the client has gone away, clean up whatever we have set up at >* connection time. >*/ > ! msg_info("disconnect from %s", state.namaddr); > smtpd_state_reset(&state); > debug_peer_restore(); > } > --- 4989,4996 >* After the client has gone away, clean up whatever we have set up at >* connection time. >*/ > ! msg_info("disconnect from %s error_count %d", > ! state.namaddr, state.error_count); > smtpd_state_reset(&state); > debug_peer_restore(); > } I would like to propose the following addition. As well as logging error_count as per the original patch, it also logs the number of messages accepted during the smtp session. The aim of that would be to identify clients that repeatedly connect and never attempt delivery. diff -ur postfix-2.11-20130623/src/smtpd/smtpd.c postfix-2.11-20130623-patch/src/smtpd/smtpd.c --- postfix-2.11-20130623/src/smtpd/smtpd.c 2013-06-23 17:10:02.0 +0200 +++ postfix-2.11-20130623-patch/src/smtpd/smtpd.c 2013-07-01 01:36:19.0 +0200 @@ -3239,9 +3239,11 @@ state->junk_cmds = 0; if (proxy) smtpd_chat_reply(state, "%s", STR(proxy->buffer)); - else + else { smtpd_chat_reply(state, "250 2.0.0 Ok: queued as %s", state->queue_id); +state->msg_count++; + } } else if (why && IS_SMTP_REJECT(STR(why))) { state->error_mask |= MAIL_ERROR_POLICY; smtpd_chat_reply(state, "%s", STR(why)); @@ -4989,7 +4991,8 @@ * After the client has gone away, clean up whatever we have set up at * connection time. */ -msg_info("disconnect from %s", state.namaddr); +msg_info("disconnect from %s error_count %d msg_count %d", +state.namaddr, state.error_count,state.msg_count); smtpd_state_reset(&state); debug_peer_restore(); } diff -ur postfix-2.11-20130623/src/smtpd/smtpd.h postfix-2.11-20130623-patch/src/smtpd/smtpd.h --- postfix-2.11-20130623/src/smtpd/smtpd.h 2012-07-30 17:53:16.0 +0200 +++ postfix-2.11-20130623-patch/src/smtpd/smtpd.h 2013-07-01 01:35:40.0 +0200 @@ -87,6 +87,7 @@ int conn_count;/* connections from this client */ int conn_rate; /* connection rate for this client */ int error_count; /* reset after DOT */ +int msg_count; /* message count */ int error_mask;/* client errors */ int notify_mask; /* what to report to postmaster */ char *helo_name; /* client HELO/EHLO argument */ diff -ur postfix-2.11-20130623/src/smtpd/smtpd_state.c postfix-2.11-20130623-patch/src/smtpd/smtpd_state.c --- postfix-2.11-20130623/src/smtpd/smtpd_state.c 2012-01-15 00:13:42.0 +0100 +++ postfix-2.11-20130623-patch/src/smtpd/smtpd_state.c 2013-07-01 01:35:59.0 +0200 @@ -86,6 +86,7 @@ state->addr_buf = vstring_alloc(100); state->conn_count = state->conn_rate = 0; state->error_count = 0; +state->msg_count = 0; state->error_mask = 0; state->notify_mask = name_mask(VAR_NOTIFY_CLASSES, mail_error_masks, var_notify_classes);
Re: PATCH: Option to log clients that execute invalid commands or disconnect with no email delivery
John Fawcett: > I would like to propose the following addition. As well as logging > error_count as per the original patch, it also logs the number of > messages accepted during the smtp session. The aim of that would be to > identify clients that repeatedly connect and never attempt delivery. Why do you even care? Remember, when the client sends no mail, then the error count reflects all rejected commands in the entire session. I don't think it is a good idea to expose random pieces of SMTP engine internals. Wietse
Re: PATCH: Option to log clients that execute invalid commands or disconnect with no email delivery
Wietse Venema: > John Fawcett: > > I would like to propose the following addition. As well as logging > > error_count as per the original patch, it also logs the number of > > messages accepted during the smtp session. The aim of that would be to > > identify clients that repeatedly connect and never attempt delivery. > > Why do you even care? Remember, when the client sends no mail, then > the error count reflects all rejected commands in the entire session. > > I don't think it is a good idea to expose random pieces of SMTP > engine internals. On the subject of sending commands without sending mail, a related counter is the "junk" command counter that is incremented with each HELO, EHLO, RSET, NOOP, VRFY, ETRN command (any command that can be repeated indefinitely without triggering an error response). Like the error counter, this counter has a limit, and it is reset after a successful mail delivery transaction. Wietse
Re: PATCH: Option to log clients that execute invalid commands or disconnect with no email delivery
On 01/07/13 02:18, Wietse Venema wrote: > John Fawcett: >> I would like to propose the following addition. As well as logging >> error_count as per the original patch, it also logs the number of >> messages accepted during the smtp session. The aim of that would be to >> identify clients that repeatedly connect and never attempt delivery. > Why do you even care? Remember, when the client sends no mail, then > the error count reflects all rejected commands in the entire session. > > I don't think it is a good idea to expose random pieces of SMTP > engine internals. > > Wietse Due to another failure of fail2ban (hopefully now solved) I had the chance to analyze another spambot attempt like the one that prompted my original post, this time 580 connections in 5 minutes. Although inexplicable to me, seems like the attack started off by connecting and disconnecting without sending any commands, so with an error_count of 0. This continued even when the concurrency limit was reached. Then from the 16th attempt the bot actually started sending AUTH commands. My idea was to detect the first attempts and block earlier, though there is not much difference in it. Another idea is to parse the log for repeated concurrency limit (smtpd_client_connection_count_limit) being reached in a small timeframe. That is already available in the log, though was probably triggered in this case because I have changed it from the default. John
Re: PATCH: Option to log clients that execute invalid commands or disconnect with no email delivery
On 01/07/13 02:59, Wietse Venema wrote: > Wietse Venema: >> John Fawcett: >>> I would like to propose the following addition. As well as logging >>> error_count as per the original patch, it also logs the number of >>> messages accepted during the smtp session. The aim of that would be to >>> identify clients that repeatedly connect and never attempt delivery. >> Why do you even care? Remember, when the client sends no mail, then >> the error count reflects all rejected commands in the entire session. >> >> I don't think it is a good idea to expose random pieces of SMTP >> engine internals. > On the subject of sending commands without sending mail, a related > counter is the "junk" command counter that is incremented with each > HELO, EHLO, RSET, NOOP, VRFY, ETRN command (any command that can > be repeated indefinitely without triggering an error response). > > Like the error counter, this counter has a limit, and it > is reset after a successful mail delivery transaction. > > Wietse I'll investigate using that since it would be non zero if mail was not delivered and zero if it was.
Re: Option to log clients that execute invalid commands or disconnect with no email delivery
On 6/28/2013 12:31 PM, John Fawcett wrote: > One type of connection which I cannot block in fail2ban are clients that > try the AUTH command on port 25, where I have disabled it. I got 245 > connections this morning in the space of 5 minutes and those are the > ones that got through despite the connection concurrency limit being hit > 277 times. In a span of 26 minutes on June 9th I got hit with parallel connections of this "AUTH bot" junk, totaling over 5,100, from a hacked Argentinian MX server. This with a concurrency limit of 4. That's just over 3 connections/second. Above you show less than 1 connect per second. ~$ zgrep " connect " /var/log/mail.log.3|grep -c 190.210.114.210 5116 ~$ zgrep "concurrency limit" /var/log/mail.log.3|grep -c 190.210.114.210 3755 Jun 9 11:01:17 greer postfix/smtpd[30422]: warning: Connection concurrency limit exceeded: 5 from mail.cqr.com.ar[190.210.114.210] for service smtp . Jun 9 11:27:43 greer postfix/smtpd[30525]: warning: Connection concurrency limit exceeded: 5 from mail.cqr.com.ar[190.210.114.210] for service smtp None were rejected. The client disconnected each time. ~$ zgrep reject /var/log/mail.log.3|grep -c 190.210.114.210 0 ~$ zgrep AUTH /var/log/mail.log.3|grep -c 190.210.114.210 1360 Looks like Anvil was responsible for the bulk of the disconnects. Anvil did its job preventing a DOS condition on smtpd. Even if these had progressed far enough to be rejected they'd still have not put significant load on the server. Thus the sum total negative impact of this attack on my MX is a bloated log. For me, personally, it's not worth the hassle to implement fail2ban simply to keep the log tidy. In your case John are you suffering anything more than a bloated log? Is one extra connect/second causing problems? -- Stan
Re: Option to log clients that execute invalid commands or disconnect with no email delivery
On 01/07/13 04:30, Stan Hoeppner wrote: > On 6/28/2013 12:31 PM, John Fawcett wrote: > >> One type of connection which I cannot block in fail2ban are clients that >> try the AUTH command on port 25, where I have disabled it. I got 245 >> connections this morning in the space of 5 minutes and those are the >> ones that got through despite the connection concurrency limit being hit >> 277 times. > > Anvil did its job preventing a DOS condition on smtpd. Even if these > had progressed far enough to be rejected they'd still have not put > significant load on the server. > > Thus the sum total negative impact of this attack on my MX is a bloated > log. For me, personally, it's not worth the hassle to implement > fail2ban simply to keep the log tidy. > > In your case John are you suffering anything more than a bloated log? > Is one extra connect/second causing problems? > I installed fail2ban more out of concerns for security across a number of services primarily sshd, but then extended to others including postfix. I then became interested in using it to block people hammering the server. I am not sure how much hammering it actual stops since I don't generally see it Only a failure of fail2ban in this case enabled me to investigate further. The additional connection load in this case is probably irrelevant, however I still prefer to block because there is no guarantee that spambots will stay within acceptable limits and I prefer to be cautious. John
Re: Bulk Mailing Performance
Bulk doesn't mean to blast the world in 1 second with emails. 1) The magic of PowerMTA consists in rotating IPs base on returned codes and returned message patterns. e.g.: if an IP addresses is banned by an ESP, will backoff on a different IP address in order in an attempt to achieve delivery. Thus, is designed for email marketing area, not for corporate email service. If you read the 330 pages guide you'll find that, by default, is sending 2 messages via 2 parallel connections. Can be increased considerable, but you need to be a genius in 'warp speed' throttling and have IPs+Sender Domains as Amazon SES has. It is very limited for inbound messages handling. 2) Postfix is a true performance MTA, used world wide (mature). The Magic of Postfix is quite complex. E.g: unlike PowerMTA, provides dynamic/adaptive throttling which is quite intelligent. It looks like it doesn't provide a way for rotating IPs as PowerMTA does. Thus, I don't see how spammers prefer Postfix. I'm still learning about Postfix secrets and how much creative can be. In my opinion, the performance for bulk deliverability should be reduce in Postfix, not increased, in order to meat ESP requirements in these days. Both MTAs are designed for two different purposes, thus, you cannot compare them. Postfix, on a *nix machine, is a true Email Server - a complex platform with many features, covering all aspects and requirements you can imagine (except the one mentioned above), but, often, many steps ahead MS Exchange. PowerMTA is an advanced sending software application for email marketers, covering exclusively their requirements and needs of rotating IPs per ESP. Marius. -- View this message in context: http://postfix.1071664.n5.nabble.com/Bulk-Mailing-Performance-tp50222p59412.html Sent from the Postfix Users mailing list archive at Nabble.com.