On 30 Jul 2016, at 7:10, Kim Roar Foldøy Hauge wrote:
I'm no expert here, but postgrey is usually a purely local test. It should terminate with a "currently busy, try again later" message very quickly.
Unless your database is very large, yes.
SPF checks and white listing require dns lookups that can potentially take much longer. Several orders of magnitude longer.
Occasionally, yes, no matter how you do DNS. However, Postfix smtpd will do multiple DNS lookups that have a strong chance of being slow before getting to any policy daemon like Postgrey. Alternatively, postscreen is a much simpler process than smtpd and in its usual config does nearly no input processing while doing DNSBL lookups in parallel, time-limited to 10s. This is usually much more efficient for dealing with spambots than going through everything that smtpd does before calling any policy daemon. Most DNSBLs worth using have robust authoritative DNS, so (unlike SPF or IP->Hostname->IP checking, both of which can require many queries) it is rarely slow to get DNSBL results.
Efficient handling of spam is all about doing the least expensive tests first in terms of cpu/time. Caching DNS can probably help a bit, but it will still require the occasional lookup now and then that take a lot longer than a good greylisting implementation should ever do.
Actually, having a local (same machine or same LAN) caching DNS resolver is extremely helpful relative to using a resolver on the other side of a router (i.e. an ISP's resolver) or worse, somewhere on the other side of 2-20 routers (i.e. OpenDNS, Google Public DNS, etc.) A lookup from a local cache typically takes <10ms, while a lookup from a public DNS server routinely will have 30-100ms of network latency built in and even one's connectivity provider's DNS will probably have at least 10ms. In any case, a DNS lookup that isn't in a resolver's cache is likely to take on the order of 100ms (but potentially multiple seconds to timeout on lookups that lead nowhere, indirectly) on top of the network latency. This leads to another reason to have a local resolver dedicated to mail servers: you can tune reply timeouts to assure that you never suffer those really long waits.
Doing an expensive test on every mail when it's not needed is badly designed setup.
Which is not really a strong argument for Postgrey or any other greylisting tool that works as as a Postfix policy daemon. The only way to spare a Postfix server from the potentially very slow client hostname DNS queries is to have postscreen in front of smtpd, at least to do pre-greet enforcement.
Many of the dns based lists also limit the amount of checks per day. Worst case scenario, you stop getting results from lists due to over use.
Yes, but those query limits are designed to make sure that commercial entities that get value from those DNSBLs and are large enough to afford participation in keeping them in operation pay for that value they receive.
If you use google's 8.8.8.8 servers for dns lookups one can quickly run into that problem, I did. A high volume of dns checks could force you into having to pay for the amount of traffic you cause.
Using Google's public DNS or anything like it assures that a mail system is low-performing and unreliable. There's nothing wrong with running an amateur mail system but one should understand that using free distant DNS run by unaccountable entities relegates a mail system to "hobbyist" class.
Many expensive network (takes a long time) checks will probably make you run out of slots a lot faster than the reconnects due to greylisting will do due to the time spent waiting for the lookups to finish.
I can only imagine that being a problem, and only on a very busy and badly misconfigured mail server. I've run systems using multiple machines each handling over a million SMTP connections per day and never had a substantial problem with doing multiple DNSBL lookups on every connection that passes a short greeting delay and SPF checks on everything that gets to the point of body filtering (i.e. SPF within SpamAssassin.)
If speed of delivery is important, you could lower the amount of time mail stays greylisted. Ideally you'd like the mail delivered the first time a server tries to send it again. If a server tries to resend once, it will most likely try more than once anyway. Having a minimum time of 300 seconds, the default of postgrey, is probably a bit excessive.
Reducing that is unlikely to reduce delays, because a lot of systems use 300s as a minimum delay between retries; for a long time the standard queue run time for Sendmail was 300s and that has been the default for Postfix since v2.4. The main exceptions using shorter retry times would be high-volume mailing lists systems (legit or otherwise) that cannot afford to let their deferral queues grow large.
