Adjust smtp to limitations of a host

[Ultrabug]

Dear list,

I'm facing a problem where I have to adapt and optimize my smtp servers
to a host's constraints which are as follow :

- maximum 3 connections to each MX of the host (he has 10 MX so
potentially I should be able to make 30 connections)
- maximum 1000 connections per MX per hour
- maximum 100 emails sent per connection

Today I'm using a special transport to match the host's domains and I
limited the transport to 3 smtp processes so I don't break the first
rule and get blocked.
In order to catch and balance between the 10 MX, I also use
smtp_mx_address_limit = 10 instead of the default 5.

My problem is that this setup is far from optimal compared to the
limitations and it slows down a lot my email delivery rate to these domains.

Anyone have any tips on how I could do this better please ?

Thanks a lot,

Regards.

Re: How to setup Postfix to Store/Forward Multi-domain + SSL to another Postfix instance?

[Wietse Venema]

dchil...@bestmail.us:
> > Postfix queues mail by default when the destination is down.
> 
> I didn't understand that from reading.  So, what triggers the redeliver
> attempt?  I'm guessing some timer/cron function in master/main config?

As required by RFC 5321 (the SMTP protocol).

http://tools.ietf.org/html/rfc5321#section-4.5.4

The Postfix SMTP client and server documentation refer to the
implemented standards, and do not include their full text.

Wietse

Re: migrating postfix setup to new server ?

[Voytek Eymont]

On Thu, March 31, 2011 12:35 am, Wietse Venema wrote:

>> what the proper (easiest?) way to migrate current setup to the new
>> server ?
>
> 1) Study the RELEASE_NOTES file and look for any incompatible
> changes that may affect your configuration.

Wietse, thanks

this seems a slightly different release, haven't found yet RELEASE_NOTES,
all I can find so far are these:

/usr/share/doc/postfix# ls

changelog.Debian.gz  copyright   README.Debian
changelog.gz NEWS.Debian.gz  test

I'll study the above, or maybe get RELEASE_NOTES from 2.7.0 source d/l on
postfix site, is that releavant ?

> 2) Make a backup of the configuration files, so you can go back.
>
>
> 3) Stop Postfix (if the RELEASE_NOTES file says you need to).
>
>
> 4) Leave the configuration files alone, install Postfix, then
> execute "postfix upgrade-configuration".
>
> 5) Start Postfix (or reload, if the difference with the old
> Postfix version does not require stopping Postfix).


I'm not clear here, this machine was given to me with Postfix 2.7.0
'pre-installed';

so, subject to RELEASE_NOTES, do I copy old#/etc/postfix/* to
new#/etc/postfix, then execute "postfix upgrade-configuration" ?

or do I copy /etc/postfix/* then (re)install Postfix over that ?


-- 
Voytek

Re: migrating postfix setup to new server ?

[Wietse Venema]

Voytek Eymont:
> I'm not clear here, this machine was given to me with Postfix 2.7.0
> 'pre-installed';

Sorry, I don't keep an up-to-date list for how to upgrade packages
for BSD version X or Y or Z, Linux distro X or Y or Z, Solaris,
and so on.

You would have saved me time if you had mentioned earlier that you
want to use the native package system.

Wietse

Re: migrating postfix setup to new server ?

[Reindl Harald]

Am 31.03.2011 14:15, schrieb Voytek Eymont:

> I'm not clear here, this machine was given to me with Postfix 2.7.0
> 'pre-installed';
> 
> so, subject to RELEASE_NOTES, do I copy old#/etc/postfix/* to
> new#/etc/postfix, then execute "postfix upgrade-configuration" ?
> 
> or do I copy /etc/postfix/* then (re)install Postfix over that ?

you should make a distribution-like package instead breaking
the package-managment, this is no solution

on fedora it is 5 minutes work install a src.rpm, replace the tarball
and after edit the SPEC-File (version, remove patches) and that was it



signature.asc
Description: OpenPGP digital signature

Re: migrating postfix setup to new server ?

[Reinaldo de Carvalho]

On Thu, Mar 31, 2011 at 10:24 AM, Reindl Harald  wrote:
>
> you should make a distribution-like package instead breaking
> the package-managment, this is no solution
>

This isn't the real problem.

> on fedora it is 5 minutes work install a src.rpm, replace the tarball
> and after edit the SPEC-File (version, remove patches) and that was it
>

Install from source or create a package have the same problem: break
distro security updates, for postfix this isn't so dangerous because
security patches aren't usual.

If you compile a software, or install not official packages, will have
a tough job to be free of vulnerabilities (public announcements occur
after the distro updates).

-- 
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

"While not fully understand a software, don't try to adapt this
software to the way you work, but rather yourself to the way the
software works" (myself)

Re: migrating postfix setup to new server ?

[Reindl Harald]

Am 31.03.2011 15:48, schrieb Reinaldo de Carvalho:
> On Thu, Mar 31, 2011 at 10:24 AM, Reindl Harald  
> wrote:
>>
>> you should make a distribution-like package instead breaking
>> the package-managment, this is no solution
>>
> 
> This isn't the real problem.

it is because a package is more than simple files

>> on fedora it is 5 minutes work install a src.rpm, replace the tarball
>> and after edit the SPEC-File (version, remove patches) and that was it
>>
> 
> Install from source or create a package have the same problem: break
> distro security updates, for postfix this isn't so dangerous because
> security patches aren't usual.

only on debian because the are flicking on old versions instead making
updates, if i make a newer version as in the distribution is newer
some weeks later their package will be installed and that is why
you should EVER make clean packages instead having old postfix from
the distri because dependencies and install another one somewhere
in the filesystem

if you make your own packages you have to make security ipdates
on your own usually, the reason for apckage-managment is not only
for this, without ANY problem you can rebuild postfix 2.8.2
for Fedora 13 and update the RPM over the 2.7.2 in a clean way
and so the migration question does not exist



signature.asc
Description: OpenPGP digital signature

Methods to limit spam sent through compromised account?

[D G Teed]

Today a user's account was compromised (likely phished) and their
credentials used to send email over our main outbound SMTP
with TLS and SASL auth.

When we learned of it, the PAM smtp configuration was set up to
block the user account authenticating and the account was soon disabled.

In the meantime, thousands of spam had gone out, as it happened
before we get to work.

Are there any suggestions on how to tune postfix to limit the spam
throughput?
There are also legitimate users who have bulk email to send, so
limiting by recipient quantity (as we do on our webmail) wouldn't be
desirable.

I've seen http://www.postfix.org/TUNING_README.html
and we are now using:

smtpd_client_connection_count_limit = 2
smtpd_client_connection_rate_limit = 10
smtpd_client_event_limit_exceptions = 127.0.0.0/8, xxx.xxx.xxx.xxx/21
smtpd_client_message_rate_limit = 10
smtpd_client_new_tls_session_rate_limit = 10
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_delay_reject = yes
anvil_rate_time_unit = 60s
anvil_status_update_time = 600s

Most of the client limits had previously been at 50 or 60.  The exceptions
line includes our servers subnet.

I'd like some idea of what real world values would be useful, or additional
suggestions
on how to make the performance less attractive to users of compromised
accounts.

I know spammers refuse to send through our webmail (another dedicated SMTP
server for that)
as they won't put up with a limit of 10 recipients.

--Donald

open relay

[Jim McIver]

Our webhosting company(which is offsite) has told me that the 
postfix-2.5 on our Freebsd 7.2 server is being used as an open relay for 
email so they have closed port 25.
We want to be able to send email from the server, but not have it relay 
for others. I've read what documentation I can find and believe I have 
it setup correctly.
Can anyone verify that the below settings should close the open 
relay...webhosting company wants verification before they will re-open 
port 25.


#postconf -n
command_directory = /usr/local/sbin
config_directory = /usr/local/etc/postfix
daemon_directory = /usr/local/libexec/postfix
data_directory = /var/db/postfix
debug_peer_level = 2
html_directory = no
inet_interfaces = loopback-only
local_transport = error:local delivery is disabled
mail_owner = postfix
mailq_path = /usr/local/bin/mailq
manpage_directory = /usr/local/man
mydestination = $myhostname, localhost.$mydomain, localhost
mydomain = lmtribune.com
mynetworks_style = host
myorigin = $mydomain
newaliases_path = /usr/local/bin/newaliases
queue_directory = /var/spool/postfix
readme_directory = no
relay_domains =
relayhost =
sample_directory = /usr/local/etc/postfix
sendmail_path = /usr/local/sbin/sendmail
setgid_group = maildrop
unknown_local_recipient_reject_code = 550

thx in advance,

--
jm

Re: open relay

[Wietse Venema]

Jim McIver:
> Our webhosting company(which is offsite) has told me that the 
> postfix-2.5 on our Freebsd 7.2 server is being used as an open relay for 
> email so they have closed port 25.
> We want to be able to send email from the server, but not have it relay 
> for others. I've read what documentation I can find and believe I have 
> it setup correctly.
> Can anyone verify that the below settings should close the open 
> relay...webhosting company wants verification before they will re-open 
> port 25.

What is their definition of open relay? There are tons of ways that
a server can be mis-used to send spam that have nothing to do with
SMTP configuration.

Wietse

users from ldap (active directory)

[vadim korsak]

Hi!

I created such ldap map file:

/etc/postfix/ldap-users.cf
server_host = 10.100.5.1
search_base = OU=Users,DC=,dc=local
version = 3
bind = yes
bind_dn = CN=mailgw,OU=SYS,DC=,DC=lan
bind_pw = password
scope = sub
result_attribute = mail
result_format = %s OK
query_filter = (&(objectClass=person)(mail=%s))

but on lookup getting error:
postmap -q  "foouser" ldap:/etc/postfix/ldap-users.cf
postmap: warning: dict_ldap_lookup: Search error 10: Referral

Adding
chase_referrals = yes
doesn't make any difference

Any ideas?

Re: open relay

[Reindl Harald]

How should they?

You do not specify any restrictions or valid addresses
This looks like a basic setup which must never see the internet

smtpd_recipient_restrictions = permit_mynetworks
 reject_non_fqdn_recipient
 reject_non_fqdn_sender
 reject_unlisted_sender
 permit_sasl_authenticated
 reject_unauth_destination
 reject_unknown_sender_domain
 reject_unknown_recipient_domain
 reject_invalid_hostname
 reject_unauth_pipelining

but you have to configure SASL-Authentication and read some manuals

Am 31.03.2011 17:28, schrieb Jim McIver:
> Our webhosting company(which is offsite) has told me that the postfix-2.5 on 
> our Freebsd 7.2 server is being used
> as an open relay for email so they have closed port 25.
> We want to be able to send email from the server, but not have it relay for 
> others. I've read what documentation I
> can find and believe I have it setup correctly.
> Can anyone verify that the below settings should close the open 
> relay...webhosting company wants verification
> before they will re-open port 25.
> 
> #postconf -n
> command_directory = /usr/local/sbin
> config_directory = /usr/local/etc/postfix
> daemon_directory = /usr/local/libexec/postfix
> data_directory = /var/db/postfix
> debug_peer_level = 2
> html_directory = no
> inet_interfaces = loopback-only
> local_transport = error:local delivery is disabled
> mail_owner = postfix
> mailq_path = /usr/local/bin/mailq
> manpage_directory = /usr/local/man
> mydestination = $myhostname, localhost.$mydomain, localhost
> mydomain = lmtribune.com
> mynetworks_style = host
> myorigin = $mydomain
> newaliases_path = /usr/local/bin/newaliases
> queue_directory = /var/spool/postfix
> readme_directory = no
> relay_domains =
> relayhost =
> sample_directory = /usr/local/etc/postfix
> sendmail_path = /usr/local/sbin/sendmail
> setgid_group = maildrop
> unknown_local_recipient_reject_code = 550
> 
> thx in advance,
> 

-- 

Mit besten Grüßen, Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / software-development / cms-solutions
p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40
icq: 154546673, http://www.thelounge.net/



signature.asc
Description: OpenPGP digital signature

example showing how to track bad/bounce emails

[marshall]

Hello;

I've been searching around for a while, but I've not found any documentation or 
examples that show how you can configure Postfix to log bad/bounce/failed 
emails 
to MySQL or how to read a log file and parse the bad/bounce/failed emails out 
of 
it.

The application I'm working on needs to detect emails that bounce and ideally 
remove them from the database. If Postfix can insert the bounced emails into a 
db table (or a log file that just contains the bad email addres, one per line), 
that would make it pretty easy to run a cron job to remove these bad emails 
from 
the application's user database.

If you know of any online guides/links, I'd appreciate it.

Thanks;

Marshall

Re: open relay

[Victor Duchovni]

On Thu, Mar 31, 2011 at 08:28:08AM -0700, Jim McIver wrote:

> Our webhosting company(which is offsite) has told me that the postfix-2.5 
> on our Freebsd 7.2 server is being used as an open relay for email so they 
> have closed port 25.

Logs of a message that failed to be rejected?

> #postconf -n
> command_directory = /usr/local/sbin
> config_directory = /usr/local/etc/postfix
> daemon_directory = /usr/local/libexec/postfix
> data_directory = /var/db/postfix
> debug_peer_level = 2
> html_directory = no
> inet_interfaces = loopback-only

With this set, and no additional SMTP listeners in master.cf, you don't
accept any external SMTP traffic on port 25, so you can't be an open
relay. However, you may have vulnerable CGI scripts that allow external
users to send email to arbitrary destinations by filling in forms...

Audit your CGI web forms.

> local_transport = error:local delivery is disabled
> mail_owner = postfix
> mailq_path = /usr/local/bin/mailq
> manpage_directory = /usr/local/man
> mydestination = $myhostname, localhost.$mydomain, localhost
> mydomain = lmtribune.com
> mynetworks_style = host
> myorigin = $mydomain
> newaliases_path = /usr/local/bin/newaliases
> queue_directory = /var/spool/postfix
> readme_directory = no
> relay_domains =
> relayhost =
> sample_directory = /usr/local/etc/postfix
> sendmail_path = /usr/local/sbin/sendmail
> setgid_group = maildrop
> unknown_local_recipient_reject_code = 550

This Postfix configuration is not an open relay.

-- 
Viktor.

SMTP client host name spoofing

[Stan Hoeppner]

Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
[66.117.14.32])
by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
for ; Thu, 31 Mar 2011 06:29:19 -0500


biz88.inmotionhosting.com is the reverse name and
mail-iw0-f176.google.com is the forward name, correct?  How is this VPS
hosted snowshoe spammer spoofing a forward host name of google.com?

I'm no DNS expert but I didn't think spoofing a forward lookup was
possible, as one must control the DNS servers (or a zone) for that
domain.  What am I missing?

-- 
Stan

Re: users from ldap (active directory)

[Victor Duchovni]

On Thu, Mar 31, 2011 at 06:36:30PM +0300, vadim korsak wrote:

> I created such ldap map file:
> 
> /etc/postfix/ldap-users.cf
> server_host = 10.100.5.1
> search_base = OU=Users,DC=,dc=local
> version = 3
> bind = yes
> bind_dn = CN=mailgw,OU=SYS,DC=,DC=lan
> bind_pw = password
> scope = sub
> result_attribute = mail
> result_format = %s OK

The result format looks wrong, perhaps you mean "OK %s", with the result
intended for potential use as an access table?

> query_filter = (&(objectClass=person)(mail=%s))
> 
> but on lookup getting error:
> postmap -q  "foouser" ldap:/etc/postfix/ldap-users.cf
> postmap: warning: dict_ldap_lookup: Search error 10: Referral

You need to use a search base that will not trigger a referral, or
use the right LDAP server. Alternatively, the LDAP server may need
to be configured to grant additional access to your "mailgw" id.

-- 
Viktor.

Re: SMTP client host name spoofing

[Victor Duchovni]

On Thu, Mar 31, 2011 at 10:52:58AM -0500, Stan Hoeppner wrote:

> Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
> [66.117.14.32])
>   by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
>   for ; Thu, 31 Mar 2011 06:29:19 -0500
> 
> 
> biz88.inmotionhosting.com is the reverse name and
> mail-iw0-f176.google.com is the forward name, correct?

No, the "google" name is just the EHLO parameter sent by the client,
it is not derived from DNS lookups and not verified.

> How is this VPS
> hosted snowshoe spammer spoofing a forward host name of google.com?

This question is moot.

-- 
Viktor.

Re: How to setup Postfix to Store/Forward Multi-domain + SSL to another Postfix instance?

[Victor Duchovni]

On Wed, Mar 30, 2011 at 10:12:40PM -0700, dchil...@bestmail.us wrote:

> I was beginning to get that idea :-(  I actually just read a coupld of
> post that you'd commented on about SNI (?), and that unless the clients
> are SNI-aware, not gonna help much.  Also DNSSEC as an option
> (someday?), but way over my head right now.
> 
> So, in addition to the SSL certs for mynet{1,2,3}.net I have a wildcard
> for *.mydomain.net.

Whatever single certificate works for you. Wildcard certs from real
CAs used to be expensive. If your cert is self-signed nobody cares
what names it contains. More typical (more affordable in most cases)
are SAN (subjectAltName) certs from real CAs that list multiple names.

> > Postfix queues mail by default when the destination is down.
> 
> I didn't understand that from reading.  So, what triggers the redeliver
> attempt?  I'm guessing some timer/cron function in master/main config?

http://www.postfix.org/OVERVIEW.html#delivering
http://www.postfix.org/QSHAPE_README.html

-- 
Viktor.

Re: Adjust smtp to limitations of a host

[Victor Duchovni]

On Thu, Mar 31, 2011 at 10:15:55AM +0200, Ultrabug wrote:

> Dear list,
> 
> I'm facing a problem where I have to adapt and optimize my smtp servers
> to a host's constraints which are as follow :
> 
> - maximum 3 connections to each MX of the host (he has 10 MX so
> potentially I should be able to make 30 connections)

What happens if you happen to exceed the limit on particular host
among the 10? If it just quickly returns a 4XX code, and does not
penalize future connections, ignore this limit and let Postfix do
what it does by default.

> - maximum 1000 connections per MX per hour

Connection caching should help if volume is high enough to worry about
this. Note this is just less than one connection every 3 seconds, but
Postfix caches idle connections for 2 seconds, so if your output rate
is 1200 messages spaced perfectly 3 seconds apart, you lose, but this
is fairly unlikely.

> - maximum 100 emails sent per connection

Postfix has no such limit, instead a connection is retired if idle
for more than 2 seconds, or after 300s (tunable). Again the site
should just return a 4XX response to RSET, and Postfix will drop
the connection and build a new connection, probably to another host.

> My problem is that this setup is far from optimal compared to the
> limitations and it slows down a lot my email delivery rate to these domains.
> 
> Anyone have any tips on how I could do this better please ?

The receiving sites policies are stupid if they don't implement
them sensibly by just returning 4XX responses without penalizing
subsequent transactions.

Have you in fact observed that default Postfix settings run into trouble
with this site? Have you considered the less aggressive concurrency
feedback controls in Postfix 2.5?

slow_initial_concurrency = 2
slow_destination_concurrency_limit = 15
slow_destination_concurrency_failed_cohort_limit = 5
slow_destination_concurrency_positive_feedback = 1/5
slow_destination_concurrency_negative_feedback = 1/8

and if absolutely necessary, in master.cf:

slow  unix  -   -   n   -   -   smtp
-o smtp_connection_reuse_time_limit=30s

(the remote side starts rejecting traffic consistently instead of
sending 421 for the 100th RSET over a given connection).

-- 
Viktor.

Re: How to setup Postfix to Store/Forward Multi-domain + SSL to another Postfix instance?

[dchilton]

Hi Wietse, Viktor,

Thanks for the references/links.

On Thu, 31 Mar 2011 12:19 -0400, "Victor Duchovni"
 wrote:
> > So, in addition to the SSL certs for mynet{1,2,3}.net I have a wildcard
> > for *.mydomain.net.
> 
> Whatever single certificate works for you. Wildcard certs from real
> CAs used to be expensive. If your cert is self-signed nobody cares
> what names it contains. More typical (more affordable in most cases)
> are SAN (subjectAltName) certs from real CAs that list multiple names.

Great, then I think I'm set.

Just for reference for other users, I've 'real' wildcard SSL certs for
$99/yr from Comodo.  A 'real' 5-cert SAN SSL from GoDaddy, great for
mixing various domains, is $80/yr.

Per your suggestion, I'm going to deploy the single-cert,
multiple-domain solution, with pre- & post-filter Postfix instances @
the edge; Zimbra on the LAN.

I'm not yet exactly sure how to best sync info/data between the various
Postfix instances ... I'm guessing that may be as simple as scp'ing
files across the net, but I'll nee to dig/read.

Thanks!

DChil

Re: Methods to limit spam sent through compromised account?

[Stan Hoeppner]

D G Teed put forth on 3/31/2011 10:21 AM:

> I'd like some idea of what real world values would be useful, or additional
> suggestions
> on how to make the performance less attractive to users of compromised
> accounts.

When you find a reasonable and effective solution to the phishing
problem please share it with the rest of us.

The only sure fire solution I'm currently aware of is mass executions of
stupid and/or ignorant people who have no business using a computer or
email.  The obvious problem with this solution is it requires
exterminating well over half the human race, including many/most of my
family members.  Not an appealing solution.

-- 
Stan

Re: SMTP client host name spoofing

[Wietse Venema]

Stan Hoeppner:
> Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
> [66.117.14.32])
>   by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
>   for ; Thu, 31 Mar 2011 06:29:19 -0500
> 

The format is:

Received: from helo-hostname (verified-reverse-name [ip-address])

This is also useful when setting up backscatter filters: all mail with
a helo-hostname of "porcupine.org", "postfix.org", etc. is a forgery.

Wietse

Re: Methods to limit spam sent through compromised account?

[Victor Duchovni]

On Thu, Mar 31, 2011 at 11:41:19AM -0500, Stan Hoeppner wrote:

> D G Teed put forth on 3/31/2011 10:21 AM:
> 
> > I'd like some idea of what real world values would be useful, or additional
> > suggestions
> > on how to make the performance less attractive to users of compromised
> > accounts.
> 
> When you find a reasonable and effective solution to the phishing
> problem please share it with the rest of us.

A good solution for large providers with many users some of whom may
have compromised accounts is per-user rate limiting, which is possible
via many of the more popular policy plugins.

-- 
Viktor.

Re: example showing how to track bad/bounce emails

[Wietse Venema]

marshall:
> Hello;
> 
> I've been searching around for a while, but I've not found any documentation 
> or 
> examples that show how you can configure Postfix to log bad/bounce/failed 
> emails 
> to MySQL or how to read a log file and parse the bad/bounce/failed emails out 
> of 
> it.
> 
> The application I'm working on needs to detect emails that bounce
> and ideally remove them from the database. If Postfix can insert
> the bounced emails into a db table (or a log file that just contains
> the bad email addres, one per line), that would make it pretty
> easy to run a cron job to remove these bad emails from the
> application's user database.

Postfix delivers mail; it does not parse bounce messages.

You could use the documented Postfix hooks to deliver the returned
mail to a command that parses out the failed recipient.

- "|command" in local alias or .forward file:
  http://www.postfix.org/local.8.html.
  http://www.postfix.org/aliases.5.html.

- Pipe-to-command daemon; this requires a per-user transprort map entry.
  http://www.postfix.org/pipe.8.html
  http://www.postfix.org/transport.5.html

Bounce message parsing is not trivial because a lot of systems use
a non-standard notification format.  Using VERP solves some but
not all of these problems (some mail systems ignore the envelope
sender address).  http://www.postfix.org/VERP_README.html

Wietse

Re: How to setup Postfix to Store/Forward Multi-domain + SSL to another Postfix instance?

[Reindl Harald]

Am 31.03.2011 18:39, schrieb dchil...@bestmail.us:

> Just for reference for other users, I've 'real' wildcard SSL certs for
> $99/yr from Comodo. 

throw them away, another two CA's from them are compromised and the
naive CTO says

"... but what we had not done was adequately consider the new (to us)
threat model of the RA being the subject of a targeted attack and
entirely compromised."

i can not remember that i ever heard a more naive argument




signature.asc
Description: OpenPGP digital signature

Re: SMTP client host name spoofing

[Stan Hoeppner]

Victor Duchovni put forth on 3/31/2011 10:57 AM:
> On Thu, Mar 31, 2011 at 10:52:58AM -0500, Stan Hoeppner wrote:
> 
>> Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
>> [66.117.14.32])
>>  by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
>>  for ; Thu, 31 Mar 2011 06:29:19 -0500
>>
>>
>> biz88.inmotionhosting.com is the reverse name and
>> mail-iw0-f176.google.com is the forward name, correct?
> 
> No, the "google" name is just the EHLO parameter sent by the client,
> it is not derived from DNS lookups and not verified.

Thanks for the clarification Viktor.  I can't seem to locate any
documentation on the official Postfix website that defines what Postfix
inserts in the first received line.  I'm sure I'm simply search
handicapped.  Can you point me the relevant docs?

Thanks.

-- 
Stan

Re: users from ldap (active directory)

[vadim korsak]

result_format = %s OK
is OK, this is checked in other places

>You need to use a search base that will not trigger a referral, or
>use the right LDAP server. Alternatively, the LDAP server may need
>to be configured to grant additional access to your "mailgw" id.

why you think this is access problem?


On Thu, Mar 31, 2011 at 6:54 PM, Victor Duchovni <
victor.ducho...@morganstanley.com> wrote:

> On Thu, Mar 31, 2011 at 06:36:30PM +0300, vadim korsak wrote:
>
> > I created such ldap map file:
> >
> > /etc/postfix/ldap-users.cf
> > server_host = 10.100.5.1
> > search_base = OU=Users,DC=,dc=local
> > version = 3
> > bind = yes
> > bind_dn = CN=mailgw,OU=SYS,DC=,DC=lan
> > bind_pw = password
> > scope = sub
> > result_attribute = mail
> > result_format = %s OK
>
> The result format looks wrong, perhaps you mean "OK %s", with the result
> intended for potential use as an access table?
>
> > query_filter = (&(objectClass=person)(mail=%s))
> >
> > but on lookup getting error:
> > postmap -q  "foouser" ldap:/etc/postfix/ldap-users.cf
> > postmap: warning: dict_ldap_lookup: Search error 10: Referral
>
> You need to use a search base that will not trigger a referral, or
> use the right LDAP server. Alternatively, the LDAP server may need
> to be configured to grant additional access to your "mailgw" id.
>
> --
>Viktor.
>

Re: example showing how to track bad/bounce emails

[marshall]

Hmm;

Thanks for your feedback, Wietse!

I'm definitely new to mail serving and not an administrator.

Would it be at all advisable to just scan the maillog file periodically for 
'status=bounce' lines and parse out the 'to<...>' email address?

It seems that'd give a pretty reasonable list of bounced emails. I don't need 
to 
be 100% accurate, just reasonably comprehensive.

Marshall





From: Wietse Venema 
To: Postfix users 
Sent: Thu, March 31, 2011 12:56:14 PM
Subject: Re: example showing how to track bad/bounce emails

marshall:
> Hello;
> 
> I've been searching around for a while, but I've not found any documentation 
> or 
>
> examples that show how you can configure Postfix to log bad/bounce/failed 
>emails 
>
> to MySQL or how to read a log file and parse the bad/bounce/failed emails out 
>of 
>
> it.
> 
> The application I'm working on needs to detect emails that bounce
> and ideally remove them from the database. If Postfix can insert
> the bounced emails into a db table (or a log file that just contains
> the bad email addres, one per line), that would make it pretty
> easy to run a cron job to remove these bad emails from the
> application's user database.

Postfix delivers mail; it does not parse bounce messages.

You could use the documented Postfix hooks to deliver the returned
mail to a command that parses out the failed recipient.

- "|command" in local alias or .forward file:
  http://www.postfix.org/local.8.html.
  http://www.postfix.org/aliases.5.html.

- Pipe-to-command daemon; this requires a per-user transprort map entry.
  http://www.postfix.org/pipe.8.html
  http://www.postfix.org/transport.5.html

Bounce message parsing is not trivial because a lot of systems use
a non-standard notification format.  Using VERP solves some but
not all of these problems (some mail systems ignore the envelope
sender address).  http://www.postfix.org/VERP_README.html

Wietse

Re: users from ldap (active directory)

[Victor Duchovni]

On Thu, Mar 31, 2011 at 08:26:17PM +0300, vadim korsak wrote:

> result_format = %s OK
> is OK, this is checked in other places
> 
> >You need to use a search base that will not trigger a referral, or
> >use the right LDAP server. Alternatively, the LDAP server may need
> >to be configured to grant additional access to your "mailgw" id.
> 
> why you think this is access problem?

Because you are getting a referral, it can be either because the search
base is wrong, or in perhaps because access is retricted. Don't expect
referrals to work, if the referral is to a different LDAP source or
if referrals require application logic (are not handled transparently
in the OpenLDAP library).

-- 
Viktor.

[4exposure...@gmail.com: Fwd: Delivery Status Notification (Failure)]

[The Doctor]

- Forwarded message from User  -

X-Original-To: postmas...@doctor.nl2k.ab.ca
Delivered-To: postmas...@doctor.nl2k.ab.ca
X-Virus-Scanned: amavisd-new at doctor.nl2k.ab.ca
Authentication-Results: doctor.nl2k.ab.ca (amavisd-new); dkim=pass
header.i=@gmail.com
Authentication-Results: doctor.nl2k.ab.ca (amavisd-new); domainkeys=pass
header.from=4exposure...@gmail.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;
h=domainkey-signature:mime-version:in-reply-to:references:date
:message-id:subject:from:to:content-type;
bh=0CzPpPHymzuAUP404uyZk1DAaLRN687Q/0hEuCwvkQk=;
b=uRmJgTcRSoqu8oEDAgrsJg7Bp70zPxNnEms9FqnuVBo8OnYuoUjlSbOwVP4MMnaBjU
gPijvDzeuJ4LLnj7LVpEOxa4u4F4s4C8pgS6sbSbMDFPzPcJaPaAtKOgg5xSz7xxukPB
PVleeL0qo8094MH73ticaTudsBRDSx3gN91Dw=
DomainKey-Signature: a=rsa-sha1; c=nofws;
d=gmail.com; s=gamma;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:content-type;
b=VhClwp8WAMFGSGOT1Dq7qm+GmYlnZ0uAM0fUeRnpzH04kl6NLlXDl4OXq1Qtht8g6S
u0jMTemKhz+MVW+GklqNEjxXBe7X55Hl0e4FfbUV4jjoim2sM0my8845Xucgi3iizGKx
4f9nBIuLH7XYppw9r5X86tRuSjZ15KyHdRGo8=
In-Reply-To: <001636e0a98b22ec89049fcaa...@google.com>
Date: Thu, 31 Mar 2011 11:35:29 -0600
Subject: Fwd: Delivery Status Notification (Failure)
From: user 
To: "Me" 
X-Sanitizer: This message has been sanitized!
X-Sanitizer-URL: http://mailtools.anomy.net/
X-Sanitizer-Rev: $Id: Sanitizer.pm,v 1.94 2006/01/02 16:43:10 bre Exp $

-- Forwarded message --
From: Mail Delivery Subsystem 
Date: Thu, Mar 31, 2011 at 11:32 AM
Subject: Delivery Status Notification (Failure)
To: u...@virt.dom.here


Delivery to the following recipient failed permanently:

root@n1

Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the recipient
domain. We recommend contacting the other email provider for further
information about the cause of this error. The error that the other server
returned was: 535 535 5.7.8 Error: authentication failed: generic failure
(SMTP AUTH failed with the remote server) (state 8).

- - Original message -

MIME-Version: 1.0
Received: by 10.142.56.18 with SMTP id e18mr317852wfa.122.1301592776704;
Thu,
 31 Mar 2011 10:32:56 -0700 (PDT)
Reply-To: user@virtual
Received: by 10.142.240.9 with HTTP; Thu, 31 Mar 2011 10:32:56 -0700 (PDT)
Date: Thu, 31 Mar 2011 11:32:56 -0600
Message-ID: 
Subject: test
From: user 
To: "Me" 
Content-Type: multipart/alternative; boundary=001636e0a98b190181049fcaaf39

- --
< Sigs supressed>

- End forwarded message -


Loooks as if gmail is trying to autheticate against the postfix
server and failing.

What is not happening?

do you need the main.cf and master.cf ? What relevant info is needed?



-- 
Member - Liberal International  This is doc...@nl2k.ab.ca Ici doc...@nl2k.ab.ca
God, Queen and country! Never Satan President Republic! Beware AntiChrist 
rising! 
http://twitter.com/rootnl2k http://www.facebook.com/dyadallee
Hey!! Hey!! ho!! Ho!! Lying Stephen Harper has got to go! on 2 May 2011 vote 
Harper out!

Re: SMTP client host name spoofing

[Stan Hoeppner]

Wietse Venema put forth on 3/31/2011 11:42 AM:
> Stan Hoeppner:
>> Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
>> [66.117.14.32])
>>  by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
>>  for ; Thu, 31 Mar 2011 06:29:19 -0500
>>
> 
> The format is:
> 
> Received: from helo-hostname (verified-reverse-name [ip-address])

Thanks Wietse.  So, answering my own previous question to Viktor, this
is defined in the docs in the backscatter readme, like so:

Although my email address is "wie...@porcupine.org", all my mail systems
announce themselves with the SMTP HELO command as
"hostname.porcupine.org". Thus, if returned mail has a Received: message
header like this:

Received: from porcupine.org ...

Thus one should deduce that the first hostname in the first received
line is the HELO/EHLO hostname.  Not quite the direct definition I
anticipated finding in the docs, or in a location I'd have expected, but
nonetheless the information is present.

> This is also useful when setting up backscatter filters: all mail with
> a helo-hostname of "porcupine.org", "postfix.org", etc. is a forgery.

I just read (again) the backscatter page.  I've never actually
implemented such measures as backscatter has never been a problem here.
 I'm thinking I'll go ahead and do so as a preemptive measure..

-- 
Stan

Re: SMTP client host name spoofing

[Victor Duchovni]

On Thu, Mar 31, 2011 at 12:20:58PM -0500, Stan Hoeppner wrote:

> > No, the "google" name is just the EHLO parameter sent by the client,
> > it is not derived from DNS lookups and not verified.
> 
> Thanks for the clarification Viktor.  I can't seem to locate any
> documentation on the official Postfix website that defines what Postfix
> inserts in the first received line.  I'm sure I'm simply search
> handicapped.  Can you point me the relevant docs?

The syntax of Received lines is specified in RFC 821/2821/5321:

http://tools.ietf.org/html/rfc5321#section-4.4

   When an SMTP server receives a message for delivery or further
   processing, it MUST insert trace ("time stamp" or "Received")
   information at the beginning of the message content, as discussed in
   Section 4.1.1.4.

   This line MUST be structured as follows:

   o  The FROM clause, which MUST be supplied in an SMTP environment,
  SHOULD contain both (1) the name of the source host as presented
  in the EHLO command and (2) an address literal containing the IP
  address of the source, determined from the TCP connection.

the details are at the bottom of page 58/top of 59:

Time-stamp-line  = "Received:" FWS Stamp 

Klensin Standards Track[Page 59]
 
RFC 5321  SMTP  October 2008

   Stamp  = From-domain By-domain Opt-info [CFWS] ";"
  FWS date-time
  ; where "date-time" is as defined in RFC 5322 [4]
  ; but the "obs-" forms, especially two-digit
  ; years, are prohibited in SMTP and MUST NOT be used.

   From-domain= "FROM" FWS Extended-Domain

   By-domain  = CFWS "BY" FWS Extended-Domain

   Extended-Domain  = Domain /
( Domain FWS "(" TCP-info ")" ) /
( address-literal FWS "(" TCP-info ")" )

   TCP-info   = address-literal / ( Domain FWS address-literal )
  ; Information derived by server from TCP connection
  ; not client EHLO.

-- 
Viktor.

Re: How to setup Postfix to Store/Forward Multi-domain + SSL to another Postfix instance?

[Victor Duchovni]

On Thu, Mar 31, 2011 at 07:15:58PM +0200, Reindl Harald wrote:

> Am 31.03.2011 18:39, schrieb dchil...@bestmail.us:
> 
> > Just for reference for other users, I've 'real' wildcard SSL certs for
> > $99/yr from Comodo. 
> 
> throw them away, another two CA's from them are compromised and the
> naive CTO says

No need to panic, many a security company has had succumbed to targetted
attack, the recent RSA Securid hack comes to mind.

Provided relying parties don't drop the Comodo root CA from their trusted
CA list, one does not gain any security from switching to a different
provider. The X.509 trusted CA model is only as strong as the weakest
CA, likely Comodo is neither strongest, nor weakest, rather they are
"in the news".

Threats to the CA infrastructure are part of the CA bargain. Verisign
issued some fake Microsoft certs some years back when an account got
broken into.

No CA is going to be perfect.

> "... but what we had not done was adequately consider the new (to us)
> threat model of the RA being the subject of a targeted attack and
> entirely compromised."
> 
> i can not remember that i ever heard a more naive argument

They'll beef up the security of the RAs, the arms-race will continue.

-- 
Viktor.

Re: Methods to limit spam sent through compromised account?

[Ralf Hildebrandt]

* D G Teed :
> Today a user's account was compromised (likely phished) and their
> credentials used to send email over our main outbound SMTP
> with TLS and SASL auth.
> 
> When we learned of it, the PAM smtp configuration was set up to
> block the user account authenticating and the account was soon disabled.
> 
> In the meantime, thousands of spam had gone out, as it happened
> before we get to work.

Well, that happens to us as well.

> Are there any suggestions on how to tune postfix to limit the spam
> throughput?
> There are also legitimate users who have bulk email to send, so
> limiting by recipient quantity (as we do on our webmail) wouldn't be
> desirable.

You probably need a policy server which limits the sender to a certain
amount of mails per time unit. If that limit is being exceeded, you
could either tempfail the mails until some human admin lifts the ban
OR put the mails on hold.

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | http://www.charite.de

Re: Methods to limit spam sent through compromised account?

[Robert Schetterer]

Am 31.03.2011 18:41, schrieb Stan Hoeppner:
> D G Teed put forth on 3/31/2011 10:21 AM:
> 
>> I'd like some idea of what real world values would be useful, or additional
>> suggestions
>> on how to make the performance less attractive to users of compromised
>> accounts.
> 
> When you find a reasonable and effective solution to the phishing
> problem please share it with the rest of us.
> 
> The only sure fire solution I'm currently aware of is mass executions of
> stupid and/or ignorant people who have no business using a computer or
> email.  The obvious problem with this solution is it requires
> exterminating well over half the human race, including many/most of my
> family members.  Not an appealing solution.
> 

using clamav-milter with sanessecurity antipish should stop
allready known spam, for sure not new ones

-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: SMTP client host name spoofing

[Stan Hoeppner]

Victor Duchovni put forth on 3/31/2011 12:44 PM:
> On Thu, Mar 31, 2011 at 12:20:58PM -0500, Stan Hoeppner wrote:
> 
>>> No, the "google" name is just the EHLO parameter sent by the client,
>>> it is not derived from DNS lookups and not verified.
>>
>> Thanks for the clarification Viktor.  I can't seem to locate any
>> documentation on the official Postfix website that defines what Postfix
>> inserts in the first received line.  I'm sure I'm simply search
>> handicapped.  Can you point me the relevant docs?
> 
> The syntax of Received lines is specified in RFC 821/2821/5321:
> 
> http://tools.ietf.org/html/rfc5321#section-4.4

No wonder I didn't find it--looking in the wrong place.

>When an SMTP server receives a message for delivery or further
>processing, it MUST insert trace ("time stamp" or "Received")
>information at the beginning of the message content, as discussed in
>Section 4.1.1.4.
> 
>This line MUST be structured as follows:
> 
>o  The FROM clause, which MUST be supplied in an SMTP environment,
>   SHOULD contain both (1) the name of the source host as presented
>   in the EHLO command and (2) an address literal containing the IP
>   address of the source, determined from the TCP connection.
> 
> the details are at the bottom of page 58/top of 59:
> 
>   Time-stamp-line  = "Received:" FWS Stamp 
> 
> Klensin Standards Track[Page 59]
>  
> RFC 5321  SMTP  October 2008
> 
>Stamp  = From-domain By-domain Opt-info [CFWS] ";"
> FWS date-time
> ; where "date-time" is as defined in RFC 5322 [4]
> ; but the "obs-" forms, especially two-digit
> ; years, are prohibited in SMTP and MUST NOT be used.
> 
>From-domain= "FROM" FWS Extended-Domain
> 
>By-domain  = CFWS "BY" FWS Extended-Domain
> 
>Extended-Domain  = Domain /
>   ( Domain FWS "(" TCP-info ")" ) /
>   ( address-literal FWS "(" TCP-info ")" )
> 
>TCP-info   = address-literal / ( Domain FWS address-literal )
> ; Information derived by server from TCP connection
> ; not client EHLO.

So the verified reverse DNS data Postfix inserts in front of the address
literal would be the "Domain folded whitespace" mentioned above or
"address-literal FWS"?  I assume the IP address in the received line
falls under "TCP-info" above.

Thanks again Viktor.

-- 
Stan

Fwd: Google The recipient server did not accept our requests to connect.

[jason hirsh]

Begin forwarded message:


From: jason hirsh 
Date: March 3, 2011 4:50:09 PM GMT-04:00
To: John Hinton 
Cc: postfix-users@postfix.org
Subject: Re: Google The recipient server did not accept our requests  
to connect.



On Mar 3, 2011, at 4:40 PM, John Hinton wrote:


On 3/3/2011 3:09 PM, jason hirsh wrote:



On Mar 3, 2011, at 4:02 PM, John Hinton wrote:


On 3/3/2011 2:52 PM, jason hirsh wrote:



On Mar 3, 2011, at 3:49 PM, John Hinton wrote:




On 3/3/2011 2:34 PM, jason hirsh wrote:




I have been informed by a couple gmail users that my server is  
blocking their access. They are getting



Technical details of temporary failure:
The recipient server did not accept our requests to connect.  
Learn more athttp://mail.google.com/support/bin/answer.py?answer=7720

[mail.kasdivi.com. (5): Connection timed out]


The wierd thing about this is that it appers to effect only a  
couple of gmail usres.   For example mail from my gmail  
account goes through just fine




Are you running a greylist perhaps? Either way, Google 'slams'  
email. In other words they try once and if the email cannot be  
delivered before the time out, the failure response is sent. I  
run milter greylist on one of my sendmail servers and had to  
include an exception for gmail. Sad but true. Gmail is not  
really a fully compliant email system but they do a lot of  
other things very good.


Yes I am running postgrey now.. but this occured even before I  
added postgrey
plus, thetre still is the issue that some accounts make it and  
some don't


There is no record of any rejection in my logs
I'm at a bit of a loss about why before postgrey. It suggests  
that the mailserver is just not answering fast enough, which can  
vary based on load at a particular moment. Gmail might have a  
short connection attempt time set as well? Obviously slamming  
reduces loads to a huge degree... shorter connection times would  
also reduce loads. It is for the most part a 'free' service.  
Perhaps premiere gmail accounts are handled diffently?




I have reviewed my logs again.. and I can find no record of any  
rejection or bounce or delay.


is there anything i can do to correct for this?

I do not know how many mails i am losing because of the gmail issue  
and I realize that it is a gmail problem

but my users don't...



As for postgrey, I have not used that but you need to add an  
exception for gmail. I assume postgrey is like milter greylist in  
that it keeps a list of recent IP addresses to allow incoming  
email. Gmail certainly has lots of IP addresses for their  
mailservers. It could be simply hit or miss based on which are  
currently on that allow list. This would show as a sporadic  
issue. Another test, if you know any of the gmail people which  
are receiving the failure notices, is to have them try once, then  
again some minutes later which is greater than you postgrey time  
setting. Then again, I'm not sure every email sent from any  
particular gmail user always come in from the same IP address.




I had this issue even before I added post grey to my  
configuratioin..   I am getting no errrors or bounces

I have had the user resend

again the wired things are
 1) one gmail account works while others don't  (which would  
support one server is really slamming while tjhe other isn't
Or some are whitelisted by postgrey while others are not. Also, is  
it possible that loads are going high at times, causing time outs?  
Gmail will just give up while almost all others simply retry later.


i have expanded my white listing in postgrey
 2) absoultely no error mesages on my server.. its like it boucned  
off an invisible shield
Do your postgrey logs show? If it is sending a delay response, it  
would only show in the logs of that 'invisible shield'. (again,  
sorry I'm out of my realm here as I only have experience with  
milter greylist on my backup sendmail server. I do remember setting  
several IP address ranges to get around gmail slamming.)


John I am out of MY realm with postgrey too...  but again the crux  
of the issue is that it occured BEFORE i had added postgrey


I have  postgrey.. logging to maillog and i have tried grep'ed  
the addresses with no find..


since it existed pre and post postgreu installation I continue to  
assume that it is something to do with postfix


can I whitelist gmail in postfix in a manner to let all thie  
rmermutations and combos go through?


I know that with all the spoofing of gmail this wull create an  
opening for spam...  I have tried to convince the user the gmail is  
a horrible thing to be using for primary mail.. but..


I understand its gmail's error...




Remember, the error message is generated by gmail, not your system.  
You could grep logs for the sender's email address.



Best of luck.

John



--
John Hinton
877-777-1407 ext 502
http://www.ew3d.com
Comprehensive Online Solutions





--
John Hinton
877-777-1407 ext 502
http://www.ew3d.com
Comprehensive O

Re: SMTP client host name spoofing

[Victor Duchovni]

On Thu, Mar 31, 2011 at 01:01:14PM -0500, Stan Hoeppner wrote:

> >Extended-Domain  = Domain /
> > ( Domain FWS "(" TCP-info ")" ) /
> > ( address-literal FWS "(" TCP-info ")" )
> > 
> >TCP-info   = address-literal / ( Domain FWS address-literal )
> >   ; Information derived by server from TCP connection
> >   ; not client EHLO.
> 
> So the verified reverse DNS data Postfix inserts in front of the address
> literal would be the "Domain folded whitespace" mentioned above or
> "address-literal FWS"?  I assume the IP address in the received line
> falls under "TCP-info" above.

Postfix takes either (really they are the same) of the

Domain FWS "(" TCP-info ")" 
address-literal FWS "(" TCP-info ")" 

branches of the spec, where the Domain or address-literal is the
EHLO name as stated earlier in 4.4. The TCP-info comment is then
of the:

Domain FWS address-literal

variety, and this time the Domain is the verified DNS name of the
address literal.

-- 
Viktor.

Re: Methods to limit spam sent through compromised account?

[pf at alt-ctrl-del.org]

"Stan Hoeppner" March 31, 2011 12:41 PM



D G Teed put forth on 3/31/2011 10:21 AM:


I'd like some idea of what real world values would be useful, or additional
suggestions
on how to make the performance less attractive to users of compromised
accounts.


When you find a reasonable and effective solution to the phishing
problem please share it with the rest of us.



You can use policyd v2 to throttle users to X mails over Y time span.

Re: Methods to limit spam sent through compromised account?

[D G Teed]

On Thu, Mar 31, 2011 at 1:41 PM, Stan Hoeppner wrote:

> D G Teed put forth on 3/31/2011 10:21 AM:
>
> > I'd like some idea of what real world values would be useful, or
> additional
> > suggestions
> > on how to make the performance less attractive to users of compromised
> > accounts.
>
> When you find a reasonable and effective solution to the phishing
> problem please share it with the rest of us.
>
> The only sure fire solution I'm currently aware of is mass executions of
> stupid and/or ignorant people who have no business using a computer or
> email.  The obvious problem with this solution is it requires
> exterminating well over half the human race, including many/most of my
> family members.  Not an appealing solution.
>

I think you misunderstood what I wrote.  I listed actual postfix
variables, looking for input on practical values.  This isn't unreasonable
to expect.  I've implemented restrictions on our webmail's SMTP
to the point that spammers login, send a few emails, find it doesn't work
for large number of recipients and then logout.  It works and I'm not
dreaming.
Horde developers have also set up limits which greatly discourage
spamming.  The only problem is we can't choke the number of
recipients on our general SMTP.

I was hoping for angles on rate limiting which normal users could live with
but would be useless for spam.  In the couple of responses so far, it seems
it requires
something external - too bad it requires more layers - I hoped it was within
the capabilities of anvil and such.

Actually, this conversation has me thinking, maybe we should have a
dedicated
SMTP just for external SMTP + TLS + SASL.  That way I can do the same
recipient limit throttling and tell people not to expect it will work for
mass
mail outs.

--Donald

Re: Adjust smtp to limitations of a host

[Mark Alan]

On Thu, 31 Mar 2011 12:39:20 -0400, Victor Duchovni
 wrote:

> The receiving sites policies are stupid if they don't implement
> them sensibly by just returning 4XX responses without penalizing
> subsequent transactions.

I am sorry to hijack this thread but we have what seems to be the
same problem.

While using the default Postfix settings (v.2.8.1 on Ubuntu 10.10), we
do have trouble to connect with several MTA's (usually
smtp1.min-saude.pt and smtp2.min-saude.pt, but sometimes others
at .min-saude.pt).
The server at smtp3.min-saude.pt never complains, nor do any of
the other email MTA at .min-saude.pt whose name do not start with
smtpNN.

When they refuse our connections, they seem to start shutting down at
25 to 30 RCPT commands, with:
"...mx postfix-slow/smtp[4907]: 36BB7818B:
to=,
relay=smtp1.min-saude.pt[194.65.151.38]:25, delay=415,
delays=414/0.25/0.41/0, dsn=4.0.0, status=deferred (host
smtp1.min-saude.pt[194.65.151.38] refused to talk to me: 421 #4.4.5 Too
many connections from your host.) "

To deal with this we are currently using:

/etc/postfix/transport
.min-saude.pt slow:

/etc/postfix/master.cf
slow  unix  -   -   -   -   -   smtp
  -o syslog_name=postfix-slow
  -o smtp_connection_cache_on_demand=no
EOT

/etc/postfix/main.cf
slow_destination_concurrency_failed_cohort_limit = 3 # we give up
after getting three 421
slow_destination_recipient_limit = 20 # keep it bellow 25
slow_destination_rate_delay = 1 # do not know if we really need this

> Have you considered the less aggressive
> concurrency feedback controls in Postfix 2.5?

Do you think that the following would be a more elegant approach than
the above described setting?

/etc/postfix/master.cf
slow  unix  -   -   -   -   -   smtp
  -o syslog_name=postfix-slow
  -o smtp_connection_reuse_time_limit=30s
EOT

/etc/postfix/main.cf
slow_initial_destination_concurrency = 2
slow_destination_concurrency_limit = 15
slow_destination_concurrency_failed_cohort_limit = 5
slow_destination_concurrency_positive_feedback = 1/5
slow_destination_concurrency_negative_feedback = 1/8

Thank you,

M.

Re: Adjust smtp to limitations of a host

[Victor Duchovni]

On Thu, Mar 31, 2011 at 07:41:41PM +0100, Mark Alan wrote:

> On Thu, 31 Mar 2011 12:39:20 -0400, Victor Duchovni
>  wrote:
> 
> > The receiving sites policies are stupid if they don't implement
> > them sensibly by just returning 4XX responses without penalizing
> > subsequent transactions.
> 
> I am sorry to hijack this thread but we have what seems to be the
> same problem.
> 
> While using the default Postfix settings (v.2.8.1 on Ubuntu 10.10), we
> do have trouble to connect with several MTA's (usually
> smtp1.min-saude.pt and smtp2.min-saude.pt, but sometimes others
> at .min-saude.pt).
> The server at smtp3.min-saude.pt never complains, nor do any of
> the other email MTA at .min-saude.pt whose name do not start with
> smtpNN.
> 
> When they refuse our connections, they seem to start shutting down at
> 25 to 30 RCPT commands, with:
> "...mx postfix-slow/smtp[4907]: 36BB7818B:
> to=,
> relay=smtp1.min-saude.pt[194.65.151.38]:25, delay=415,
> delays=414/0.25/0.41/0, dsn=4.0.0, status=deferred (host
> smtp1.min-saude.pt[194.65.151.38] refused to talk to me: 421 #4.4.5 Too
> many connections from your host.) "

Why would this be a response to "too many recipient commands", a single
message with many recipients is sent over a single connection, unless
you have set an ill-advised destination recipient limit.

> To deal with this we are currently using:
> 
> /etc/postfix/transport
> .min-saude.pt slow:
> 
> /etc/postfix/master.cf
> slow  unix  -   -   -   -   -   smtp
>   -o syslog_name=postfix-slow
>   -o smtp_connection_cache_on_demand=no
> EOT
> 
> /etc/postfix/main.cf
> slow_destination_concurrency_failed_cohort_limit = 3 # we give up
> after getting three 421
> slow_destination_recipient_limit = 20 # keep it bellow 25

This increases the number of connections, which is unlikely what you
want, provided of course you have messages with a large recipient count.

> slow_destination_rate_delay = 1 # do not know if we really need this

This limits you to one connection at-a-time.

> > Have you considered the less aggressive
> > concurrency feedback controls in Postfix 2.5?
> 
> Do you think that the following would be a more elegant approach than
> the above described setting?
> 
> /etc/postfix/master.cf
> slow  unix  -   -   -   -   -   smtp
>   -o syslog_name=postfix-slow
>   -o smtp_connection_reuse_time_limit=30s
> EOT
> 
> /etc/postfix/main.cf
> slow_initial_destination_concurrency = 2
> slow_destination_concurrency_limit = 15
> slow_destination_concurrency_failed_cohort_limit = 5
> slow_destination_concurrency_positive_feedback = 1/5
> slow_destination_concurrency_negative_feedback = 1/8

That depends on how determined the remote site is to damage the
SMTP eco-system by imposing counter-productive punitive mechanisms
on legitimate senders. You can certainly try, and report your findings.

-- 
Viktor.

Re: virtual_alias_maps and recipient_delimiter

[Jeroen Geilman]

On 03/31/2011 08:41 AM, Corey Quinn wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


On Mar 30, 2011, at 2:08 PM, Noel Jones wrote:

   

On 3/30/2011 3:53 PM, Ansgar Wiechers wrote:
 

On 2011-03-30 Corey Quinn wrote:
   

On Mar 30, 2011, at 12:46 PM, Noel Jones wrote:
 

# main.cf
smtp_generic_maps =
  regexp:/etc/postfix/generic.regexp

# generic.regexp
IF /+.*@example\.com$/
/^(.*)+/  $1...@example.com
ENDIF
   

Threw this verbatim into my generic.regexp:

[root@mx1 postfix]# postmap -q "user+t...@example.com" regexp:generic.regexp
postmap: warning: regexp map generic.regexp, line 1: Invalid preceding regular 
expression
postmap: warning: regexp map generic.regexp, line 4: ignoring ENDIF without 
matching IF
user+t...@example.com@example.com

On the plus side, I figured out how to do something interesting by
reading through the regexp documentation-- this solved another
challenge I'd been facing.
 

'+' has a special meaning in regular expressions (one or more times the
preceding term), so you need to escape it to match a literal '+':

if /\+.*@example\.com$/
/^(.*)\+/ $1...@example.com
endif

Regards
Ansgar Wiechers
   


that's what I get for posting an untested example and then walking away for a 
little while.

Escaping the "+" fixes the expression to what I intended.

Thanks everyone for cleaning up after me.

 

Thanks-- that sorted it, and now I'm able to say I learned two new things about 
regular expressions today.

Only several thousand left to go.
   


I don't know if this is limited to PCRE or usable in regexp too, but the 
above would match joefoo, and return joe+++ due to the greedy (.*).


To avoid such edge-cases:

/^([^+]+)\+[^+@]+@example.com/$1...@example.com

That is, one or more of not a plus sign, followed by a plus sign, 
followed by one or more of not a plus nor an @.


This would capture the first alpha-numeric sequence not including a 
plus, and completely ignore non-delimited addresses (since they don't 
contain a plus sign) and odd addresses that contain pluses but aren't 
delimited by one.



--
J.

Re: SMTP client host name spoofing

[Jeroen Geilman]

On 03/31/2011 07:41 PM, Stan Hoeppner wrote:

Wietse Venema put forth on 3/31/2011 11:42 AM:
   

Stan Hoeppner:
 

Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
[66.117.14.32])
by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
for; Thu, 31 Mar 2011 06:29:19 -0500

   

The format is:

 Received: from helo-hostname (verified-reverse-name [ip-address])
 

Thanks Wietse.  So, answering my own previous question to Viktor, this
is defined in the docs in the backscatter readme, like so:

Although my email address is "wie...@porcupine.org", all my mail systems
announce themselves with the SMTP HELO command as
"hostname.porcupine.org". Thus, if returned mail has a Received: message
header like this:

 Received: from porcupine.org ...

Thus one should deduce that the first hostname in the first received
line is the HELO/EHLO hostname.  Not quite the direct definition I
anticipated finding in the docs, or in a location I'd have expected, but
nonetheless the information is present.

   

This is also useful when setting up backscatter filters: all mail with
a helo-hostname of "porcupine.org", "postfix.org", etc. is a forgery.
 

I just read (again) the backscatter page.  I've never actually
implemented such measures as backscatter has never been a problem here.
  I'm thinking I'll go ahead and do so as a preemptive measure..

   
Backscatter can be a HUGE problem, especially when spammers send you 
bounces (with the empty mailer-daemon sender address <>), since you MUST 
accept those.


HELO checks are the primary defense against backscatter of this sort; I 
use a simple subset of the available options:


smtpd_helo_restrictions = reject_invalid_helo_hostname, 
reject_unknown_helo_hostname, reject_non_fqdn_helo_hostname, 
check_helo_access hash:/etc/postfix/helo_access, permit


Where helo_access contains my own IPs and hostnames.

This setup will reject an AMAZING amount of spam.
Fair warning: it may also yield the occasional false positive due to a 
misconfigured client mail system!

The usual warn_if_reject will help out with that.

--
J.

Postscreen + Logwatch = A bunch of unmatched entries

[Steve Jenkins]

Ever since turning on Postscreen (which I love), my nightly LogWatch
reports (running 7.3.6) have bunches of unmatched entries due to
Postscreen. Anyone know if LogWatch 7.4.0 recognizes them, or how to
configure it so that I get usable Postscreen stats?

Thanks,

SteveJ

Re: Methods to limit spam sent through compromised account?

[D G Teed]

On Thu, Mar 31, 2011 at 3:34 PM, pf at alt-ctrl-del.org  wrote:

>
> "Stan Hoeppner" March 31, 2011 12:41 PM
>
>  D G Teed put forth on 3/31/2011 10:21 AM:
>>
>>  I'd like some idea of what real world values would be useful, or
>>> additional
>>> suggestions
>>> on how to make the performance less attractive to users of compromised
>>> accounts.
>>>
>>
>> When you find a reasonable and effective solution to the phishing
>> problem please share it with the rest of us.
>>
>>
> You can use policyd v2 to throttle users to X mails over Y time span.
>
>
Thanks.  This might be the way to go as it also would address
the case of local users if their system is compromised.

Would you happen to know how it behaves when the SQL backend
isn't available?  I know sqlgrey keeps working OK if the DB is
unreachable.

--Donald

Re: Methods to limit spam sent through compromised account?

[Jeroen Geilman]

On 03/31/2011 08:36 PM, D G Teed wrote:



On Thu, Mar 31, 2011 at 1:41 PM, Stan Hoeppner > wrote:


D G Teed put forth on 3/31/2011 10:21 AM:

> I'd like some idea of what real world values would be useful, or
additional
> suggestions
> on how to make the performance less attractive to users of
compromised
> accounts.

When you find a reasonable and effective solution to the phishing
problem please share it with the rest of us.

The only sure fire solution I'm currently aware of is mass
executions of
stupid and/or ignorant people who have no business using a computer or
email.  The obvious problem with this solution is it requires
exterminating well over half the human race, including many/most of my
family members.  Not an appealing solution.


I think you misunderstood what I wrote.  I listed actual postfix
variables, looking for input on practical values.


That depends way too much on your particular setup.


This isn't unreasonable
to expect.  I've implemented restrictions on our webmail's SMTP
to the point that spammers login, send a few emails, find it doesn't work
for large number of recipients and then logout.


For such scenarios, even tighter restrictions (enforced by methods 
external to postfix) would seem appropriate: simply record the number of 
failed deliveries or spamassassin hits per web user, and disable the 
account if this exceeds some threshold.
Rest assured that the big webmail providers use quite advanced methods 
to detect hacked mail accounts that try to send spam - and they don't 
depend solely on the MTA to provide this protection.



It works and I'm not dreaming.
Horde developers have also set up limits which greatly discourage
spamming.  The only problem is we can't choke the number of
recipients on our general SMTP.
I was hoping for angles on rate limiting which normal users could live 
with but would be useless for spam.  In the couple of responses so 
far, it seems it requires
something external - too bad it requires more layers - I hoped it was 
within

the capabilities of anvil and such.


Rate limiting is certainly within anvil's capabilities, but you don't 
want to constrain this on your external smtpd(8) listener.

Use submission instead.

Submission cleanly separates incoming mail from the Internet from 
submissions by your own users, with no chance of the two types of 
traffic mingling.
A separate smtpd(8) listener purely for webmail is also a good idea - 
this doesn't even have to listen on the network.


All of these can have distinct restrictions placed on the traffic they 
accept.


Actually, this conversation has me thinking, maybe we should have a 
dedicated

SMTP just for external SMTP + TLS + SASL.  That way I can do the same
recipient limit throttling and tell people not to expect it will work 
for mass

mail outs.


Yes: it's called "submission"; it runs on port 587 by default and was 
invented for this purpose.
You could also provide the people that need mass mailing with "special" 
accounts that have less strict limits.


--
J.

Re: Postscreen + Logwatch = A bunch of unmatched entries

[Steve Jenkins]

On Thu, Mar 31, 2011 at 12:29 PM, Steve Jenkins  wrote:
> Anyone know if LogWatch 7.4.0 recognizes them

Well, I can answer my first question myself. I just installed it and
can confirm that Logwatch 7.4.0 (released earlier this month) does NOT
recognize Postscreen entries:

 **Unmatched Entries**

 CONNECT from [127.0.0.1]:39527
 WHITELISTED [127.0.0.1]:39527
 CONNECT from [66.34.88.172]:33743
 addr 66.34.88.172 listed by domain b.barracudacentral.org as 127.0.0.2
 DNSBL rank 2 for [66.34.88.172]:33743
 NOQUEUE: reject: RCPT from [66.34.88.172]:33743: 550 5.7.1 Service
unavailable; client [66.34.88.172] blocked using
b.barracudacentral.org; from=,
to=, proto=ESMTP, helo=
 DISCONNECT [66.34.88.172]:33743

Anyone had any luck getting Postscreen and Logwatch to play nice together?

SteveJ

Re: example showing how to track bad/bounce emails

[Wietse Venema]

marshall:
> If Postfix can insert the bounced emails into a db table (or a
> log file that just contains the bad email addres, one per line),
> that would make it pretty easy to run a cron job to remove these
> bad emails from the application's user database.

Wietse:
> You could use the documented Postfix hooks to deliver the returned
> mail to a command that parses out the failed recipient.
[examples deleted]

marshall:
> Would it be at all advisable to just scan the maillog file periodically for 
> 'status=bounce' lines and parse out the 'to<...>' email address?

Sorry, email is not an end-to-end protocol like HTTP. It is a store
and forward protocol where mail travels over multiple SMTP sessions.

Logfile parsing gives you only information about the first-hop mail
delivery attempt.  It does not give information about errors AFTER
the first hop.

Wietse

Re: Methods to limit spam sent through compromised account?

[lst_hoe02]

Zitat von D G Teed :


On Thu, Mar 31, 2011 at 1:41 PM, Stan Hoeppner wrote:


D G Teed put forth on 3/31/2011 10:21 AM:

> I'd like some idea of what real world values would be useful, or
additional
> suggestions
> on how to make the performance less attractive to users of compromised
> accounts.

When you find a reasonable and effective solution to the phishing
problem please share it with the rest of us.

The only sure fire solution I'm currently aware of is mass executions of
stupid and/or ignorant people who have no business using a computer or
email.  The obvious problem with this solution is it requires
exterminating well over half the human race, including many/most of my
family members.  Not an appealing solution.



I think you misunderstood what I wrote.  I listed actual postfix
variables, looking for input on practical values.  This isn't unreasonable
to expect.  I've implemented restrictions on our webmail's SMTP
to the point that spammers login, send a few emails, find it doesn't work
for large number of recipients and then logout.  It works and I'm not
dreaming.
Horde developers have also set up limits which greatly discourage
spamming.  The only problem is we can't choke the number of
recipients on our general SMTP.

I was hoping for angles on rate limiting which normal users could live with
but would be useless for spam.  In the couple of responses so far, it seems
it requires
something external - too bad it requires more layers - I hoped it was within
the capabilities of anvil and such.

Actually, this conversation has me thinking, maybe we should have a
dedicated
SMTP just for external SMTP + TLS + SASL.  That way I can do the same
recipient limit throttling and tell people not to expect it will work for
mass
mail outs.


If you don't like rate-limits you might go with content-filter on  
*outgoing* mail and instead of inserting headers or dropping the mail  
put mail on hold and disable the account in question if some  
"spaminess" limit is reached. This no easy lightwight/solution either  
but you can get away without rate-limits.


Regards

Andreas




smime.p7s
Description: S/MIME Cryptographic Signature

Re: Adjust smtp to limitations of a host

[Mark Alan]

On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni
 wrote:
> Why would this be a response to "too many recipient commands", a
> single message with many recipients is sent over a single connection,
> unless you have set an ill-advised destination recipient limit.

All _recipient_limit parameters are all at their defaults. With the
exception of things related to ciphers and TLS, we try hard to keep the
default Postfix settings.

> > /etc/postfix/main.cf
> > slow_destination_concurrency_failed_cohort_limit = 3 # we give up
> > after getting three 421
> > slow_destination_recipient_limit = 20 # keep it bellow 25
> 
> This increases the number of connections, which is unlikely what you
> want, provided of course you have messages with a large recipient
> count.

It was not obvious to us. The idea was simply to put a limit on each
burst of messages sent to the slow transport MTA's.

These messages are related to a low traffic (2-3 messages a month), low
volume (280 subscribers) mailing list, managed with mlmmj and using
VERP tagging.
We have exactly 142 subscribers from subdomains at .min-saude.pt.
Hardly huge numbers.

> > slow_destination_rate_delay = 1 # do not know if we really need this
> This limits you to one connection at-a-time.

The idea was to have a 1s delay between each message delivered. But, of
course not knowing if this helped or not.

> > /etc/postfix/master.cf
> > slow  unix  -   -   -   -   -   smtp
> >   -o syslog_name=postfix-slow
> >   -o smtp_connection_reuse_time_limit=30s

Should we use only those 2 lines, or should we also add
  -o smtp_connection_cache_on_demand=no

> > /etc/postfix/main.cf
> > slow_initial_destination_concurrency = 2
> > slow_destination_concurrency_limit = 15
> > slow_destination_concurrency_failed_cohort_limit = 5
> > slow_destination_concurrency_positive_feedback = 1/5
> > slow_destination_concurrency_negative_feedback = 1/8
> 
> That depends on how determined the remote site is to damage the
> SMTP eco-system by imposing counter-productive punitive mechanisms
> on legitimate senders.

Being it the health ministry bureaucracy, I am pretty sure that they
have the time and resources to be creative at it.
We know for sure that up until now they did not answer any emails regarding 
their strange
mail server policies.

> You can certainly try, and report your

We will wait for your opinion on the above
-o smtp_connection_cache... parameter, to try to those new settings.


Thank you,

M.

Re: SMTP client host name spoofing

[Stan Hoeppner]

Jeroen Geilman put forth on 3/31/2011 2:16 PM:

> Backscatter can be a HUGE problem, especially when spammers send you
> bounces (with the empty mailer-daemon sender address <>), since you MUST
> accept those.

Spammers don't send backscatter bounces.  The victim MX hosts do, by
definition.  In this case I've not seen any for many years.

Backscatter once was a HUGE problem, but not today.  As most, if not
all, forged sender address spam comes from bots, and most MXen on the
planet today are far better equipped to reject bot spam, the rest of us
don't suffer the backscatter as we once did.

In the case of spammers directly sending spam disguised as fake
mailer-daemon messages, those are dealt with here via the same methods I
use against all other spam, which is probably why I never see them these
days--blocked at SMTP time.

Finally, snowshoe spammers don't bother to forge valid sender addresses,
as that requires extra work--most spammers are lazy.  Snowshoe spammers
use throw away domains, and simply make up random sender addresses in
that domain.

-- 
Stan

Re: SMTP client host name spoofing

[mouss]

Le 31/03/2011 17:52, Stan Hoeppner a écrit :
> 
> Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
> [66.117.14.32])
>   by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
>   for ; Thu, 31 Mar 2011 06:29:19 -0500
> 
> 
> biz88.inmotionhosting.com is the reverse name and
> mail-iw0-f176.google.com is the forward name, correct?  How is this VPS
> hosted snowshoe spammer spoofing a forward host name of google.com?
> 

they are spoofing HELO. if you feel motivated, contact InMotion.
otherwise, block list
66.117.14.0 - 66.117.15.255

or block

/\d\.inmotionhosting\.com$/ REJECT


and complain to "Corporate Colocation Inc.". if no answer, block:
66.117.0.0 - 66.117.15.255


> I'm no DNS expert but I didn't think spoofing a forward lookup was
> possible, as one must control the DNS servers (or a zone) for that
> domain.  What am I missing?
> 

Re: Postscreen + Logwatch = A bunch of unmatched entries

[Sahil Tandon]

On Thu, 2011-03-31 at 12:50:30 -0700, Steve Jenkins wrote:

> On Thu, Mar 31, 2011 at 12:29 PM, Steve Jenkins  
> wrote:
> > Anyone know if LogWatch 7.4.0 recognizes them
> 
> Well, I can answer my first question myself. I just installed it and
> can confirm that Logwatch 7.4.0 (released earlier this month) does NOT
> recognize Postscreen entries:

[ .. ]

> Anyone had any luck getting Postscreen and Logwatch to play nice together?

Have you tried asking the creators of the log parsing software?

-- 
Sahil Tandon 

Re: SMTP client host name spoofing

[Stan Hoeppner]

mouss put forth on 3/31/2011 4:38 PM:
> Le 31/03/2011 17:52, Stan Hoeppner a écrit :
>>
>> Received: from mail-iw0-f176.google.com (biz88.inmotionhosting.com
>> [66.117.14.32])
>>  by greer.hardwarefreak.com (Postfix) with ESMTP id F297D6C12E
>>  for ; Thu, 31 Mar 2011 06:29:19 -0500
>>
>>
>> biz88.inmotionhosting.com is the reverse name and
>> mail-iw0-f176.google.com is the forward name, correct?  How is this VPS
>> hosted snowshoe spammer spoofing a forward host name of google.com?
>>
> 
> they are spoofing HELO. 

Which is the answer to my question.

> if you feel motivated, contact InMotion.
> otherwise [snip]

You should know me well enough by now mouss to realize that I'd already
blocked the parent /20, and Corporate-Colocation's other 19 netblocks,
after some investigation, before I sent my question to the list. ;)

-- 
Stan

Re: Postscreen + Logwatch = A bunch of unmatched entries

[Wietse Venema]

Sahil Tandon:
> On Thu, 2011-03-31 at 12:50:30 -0700, Steve Jenkins wrote:
> 
> > On Thu, Mar 31, 2011 at 12:29 PM, Steve Jenkins  
> > wrote:
> > > Anyone know if LogWatch 7.4.0 recognizes them
> > 
> > Well, I can answer my first question myself. I just installed it and
> > can confirm that Logwatch 7.4.0 (released earlier this month) does NOT
> > recognize Postscreen entries:
> 
> [ .. ]
> 
> > Anyone had any luck getting Postscreen and Logwatch to play nice together?
> 
> Have you tried asking the creators of the log parsing software?

I suppose it is a script with a config file with patterns for "known"
logfile messages. If someone can share a copy, then I could try to
write some rules for normal postscreen, dnsblog and tlsproxy logging.

Wietse

Re: Adjust smtp to limitations of a host

[Victor Duchovni]

On Thu, Mar 31, 2011 at 10:18:52PM +0100, Mark Alan wrote:

> On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni
>  wrote:
> > Why would this be a response to "too many recipient commands", a
> > single message with many recipients is sent over a single connection,
> > unless you have set an ill-advised destination recipient limit.
> 
> All _recipient_limit parameters are all at their defaults. With the
> exception of things related to ciphers and TLS, we try hard to keep the
> default Postfix settings.
> 
> > > /etc/postfix/main.cf
> > > slow_destination_concurrency_failed_cohort_limit = 3 # we give up
> > > after getting three 421
> > > slow_destination_recipient_limit = 20 # keep it bellow 25
> > 
> > This increases the number of connections, which is unlikely what you
> > want, provided of course you have messages with a large recipient
> > count.
> 
> It was not obvious to us. The idea was simply to put a limit on each
> burst of messages sent to the slow transport MTA's.

The actual effect is to drive up the number of messages, this parameter
limits the number of recipients per message delivery, thus a 100-recipient
message now gets sent 5 times instead of the default 2.

> > > /etc/postfix/master.cf
> > > slow  unix  -   -   -   -   -   smtp
> > >   -o syslog_name=postfix-slow
> > >   -o smtp_connection_reuse_time_limit=30s
> 
> Should we use only those 2 lines, or should we also add
>   -o smtp_connection_cache_on_demand=no

Connection caching can help reduce connection setup cost, especially
when using rate delays, though you are then keeping an open connection
needlessly idle every other second. If connectivity to the MX hosts
in question is reliable (say they are using load balancers, ...)
connection caching could be turned off...

> > > /etc/postfix/main.cf
> > > slow_initial_destination_concurrency = 2
> > > slow_destination_concurrency_limit = 15
> > > slow_destination_concurrency_failed_cohort_limit = 5
> > > slow_destination_concurrency_positive_feedback = 1/5
> > > slow_destination_concurrency_negative_feedback = 1/8
> > 
> > That depends on how determined the remote site is to damage the
> > SMTP eco-system by imposing counter-productive punitive mechanisms
> > on legitimate senders.
> 
> Being it the health ministry bureaucracy, I am pretty sure that they
> have the time and resources to be creative at it.

:-)

> We know for sure that up until now they did not answer any emails regarding 
> their strange
> mail server policies.
> 
> > You can certainly try, and report your
> 
> We will wait for your opinion on the above
> -o smtp_connection_cache... parameter, to try to those new settings.

With rate delays in place, disabling connection caching may spread
the load more evenly, but will reduce performance when one of the
MX hosts is down. It's a trade-off.

If you can take a small risk, you may be able to achieve reasonable
throughput with less hand-tuning, by letting the more advanced feedback
mechanisms in Postfix adjust the site's negative replies. This depends
on the site's feedback being reasonably timely and the consequences of
"overshoot" being transient.

-- 
Viktor.