On 8/27/2013 5:01 PM, Jeroen Geilman wrote:
> On 08/25/2013 08:11 PM, Niclas Arndt wrote:
>> Sorry if this is slightly off-topic, but at least a bunch of experts
>> are listening.
>>
>> I am using Spamhaus (and other methods) and over time I have amassed a
>> list of IP ranges that (according to Spamhaus) shouldn't be sending
>> e-mail at all.
Given your description this would be the PBL list, typically filled with
consumer broadband IP ranges.
>> One problem is that this list tends to become quite
>> long and another is that I would like to verify it so that I don't
>> eventually block legitimate e-mail.
This suggests you've collected these IPs from your logs and have built a
local table. Why? I don't understand what you mean by "verify it".
WRT blocking legitimate email that's always a risk with dnsbls, though
in the case of the Spamhaus lists a very low risk.
>> On the other hand, I would like to place as little a load as possible
>> on Spamhaus.
See below. But note you are allowed 300K queries/day to Zen.
>> Here are my questions: Is the iptables approach at all viable in the
>> long run? Is there any non-commercial way to upload a text file
>> containing spamming IP addresses and have it verified for correctness?
>
> postfix 2.8 and later offer the postscreen(8) triage service, which
> deals very efficiently with large amounts of DNSBL lookups.
> Run a local DNS cache on the postfix machine and point postscreen at zen.
> You'll be hitting the spamhaus non-commercial limit long before you hit
> the local cache's limits.
>
> This automatically adds and expires DNSBL entries without any effort
> from you, as a free bonus (this is the biggest problem with your
> iptables approach.)
A local caching resolver is a good start. I use pdns_recursor.
If you're concerned about your query load to Spamhaus public Zen servers
then block spam connections without using dnsbls, or more precisely,
block as much as possible before querying dnsbls. This is what myself
and many others do. I.e. use the least expensive tools first and the
most expensive last. Expense being defined as time to completion and
resources consumed (net/CPU/RAM/disk IO).
This not only decreases the load you put on dnsbls, but it also
increases throughput due to decreasing remote lookup latency per
message. Judicious use of inbuilt Postfix features and local tables can
cut per msg latency from a few dozen milliseconds down to a few
microseconds.
To do this, enable Postcreen in its standard mode. Do not configure
dnsbl block lists in postscreen. This will stop nearly all bot traffic.
Next, use the "everything under smtpd_recipient_restrictions" main.cf
model so you can precisely control the order in which restrictions are
processed. Inbuilt Postfix restrictions are the least expensive, then
CDB, hash, and CIDR tables. REGEXP and PCRE tables are a little more
expensive, possibly much more if tables are large (and without loops)
and expressions complex. Header checks are normally next in expense
followed by body checks, and local policy daemons and content filters
can be more expensive still.
So as a general starting point you'd want something similar to this.
This configuration assumes the Postfix server is an MX as well as a
submission relay.
smtpd_recipient_restrictions =
# These are inbuilt Postfix restrictions
permit_mynetworks
permit_sasl_authenticated
reject_unauth_destination
reject_unknown_reverse_client_hostname
reject_non_fqdn_sender
reject_non_fqdn_helo_hostname
reject_invalid_helo_hostname
reject_unlisted_recipient
# These check a hash table of domains which have sent spam here
# or that have been obtained via intelligence (A/S mailing lists)
# and check dnswl.org for sender that should be allowed through
check_reverse_client_hostname_access hash:/etc/postfix/blacklist
check_helo_access hash:/etc/postfix/blacklist
check_sender_access hash:/etc/postfix/blacklist
check_client_access hash:/etc/postfix/whitelist
permit_dnswl_client list.dnswl.org=127.0.[2..14].[2..3]
# This checks a rather larger CIDR table of snowshoe spammer netblocks
# that I've been building for ~4 years now
check_client_access /etc/postfix/spammer
# If none of the previous restrictions reject the spam I then query
# various [dns|rhs]bls.
reject_rbl_client zen.spamhaus.org
reject_rbl_client b.barracudacentral.org
reject_rbl_client psbl.surriel.com
reject_rhsbl_reverse_client dbl.spamhaus.org
reject_rhsbl_sender dbl.spamhaus.org
reject_rhsbl_helo dbl.spamhaus.org
If you want to be able to score [dns|rhs]bl results you can use a policy
filter here such as postfwd or policyd instead of the simple
reject/allow offered by these Postfix restrictions.
Combining this with a local caching DNS resolver as Jeroen suggested you
should cut down on your dnsbl query load pretty substantially.
Now, regarding the list you've already built, if you would like I can
provide you a tool that will automatically convert it into a Postfix
CIDR table that you can use with check_client_access. Though if you
built that list in the manner I'm guessing, then I'd recommend against
it. Some of the IPs may be legit systems that were temporarily listed
due to malware outbreak or similar, and are permanently in your file.
--
Stan
