Hi,
On Oct 27, 2010, at 11:50 PM, Noel Jones wrote:
On 10/27/2010 7:02 PM, Al Zick wrote:
Is there a replacement for procmail? I know it seemed to take
longer and did raise cpu usage, but when I first installed it
with bogofilter, it almost eliminated spam getting into my inbox.
depends on why you're using procmail... If you need a way to
interface spam/virus filtering, amavisd-new + spamassassin + clamav
+ sanesecurity clam signatures are a popular and effective
combination, although SpamAssassin can use quite a bit of resources.
Currently, I just use procmail to interface with the spam filters. I
would really like to put a bunch of rules into procmail too, for
example: if is sees the word viagra anywhere in the email, it is
spam, there is no reason to go any further with it.
Right now, I am concerned that I would need a quad core, quad
processor system that was dedicated to just running spamassassin, so
I am looking at other solutions.
problems lately have been with email. I feel like I need to
get postfix to stop using so much cpu.
Show some evidence. Postfix shouldn't use very much CPU.
per second hitting the mail server just to be temporarily
bounced by the graylisting when in the end they get bounced
anyway. Even after they are bounced, they just keep coming
anyway.
Most greylist services use DEFER_IF_PERMIT so that mail that can be
permanently rejected is not deferred to retry.
I think that I need to accept and delete email that is being sent to
maybe the top few email address that don't exist and never had
existed. They add the most lines to the log. When I was just
accepting them and deleting them, then the log was very quiet.
If your forwarded mail is what's attempting repeated delivery
despite being rejected, you'll need to whitelist those servers and
eat the mail. Otherwise, firewall clients who refuse to go away.
I will definitely be whitelisting all the servers that forward email
to me. I will also be whitelisting all my friend's mail servers. This
will probably help with a lot of the bounce rebouncing.
Identify the problem, then address it
Sounds as if you've foolishly set "soft_bounce = yes"
# postconf -d | grep soft_bounce
soft_bounce = no
"man postconf" to see what "-d" does and why the above information
is useless.
But no matter; soft_bounce doesn't appear in your "postconf -n"
listing, so that's not it.
Is there anything else that could cause a soft_bounce?
[postconf output]
bounce_queue_lifetime = 2d
default_destination_concurrency_limit = 5
default_process_limit = 15
maximal_backoff_time = 4h
maximal_queue_lifetime = 3d
minimal_backoff_time = 2h
qmgr_message_active_limit = 50
qmgr_message_recipient_limit = 50
queue_run_delay = 30m
Your settings resemble what someone with an underpowered server
with a bad backscatter problem might try. If that's not your
situation, use the defaults. If that *is* your situation, address
the source of the problem rather than putting postfix colored band-
aids on it.
What exactly is a backscatter problem?
If I do have a backscatter problem, what should the settings be?
Mucking around with the above settings is a good way to cripple
postfix performance. Tread carefully here.
With a process limit of 15, any server less than 10 years old
should hardly get above idle. The default has been 100 for years;
most servers can easily support several times that.
This install of postfix is from a few years ago and it was not up to
date then (it is what installed with the OS and I never updated it).
A friend of mine recompiled OS for better optimization. I think it
was already pretty old when I install it. Really, I was supposed to
upgrade Postfix through the packaging system because there was some
known problems with what came with the OS, but I never did. I had a
friend of mine look at it because it would not receive or send emails
to the outside world, and I am not really sure what he did anymore. I
think he added one line to master.cf and I think he had me make other
changes to master.cf (although, he may have made them). I do remember
that the server would basically not work at all and I think the
process limit was set to something lower and I raised it to 15. This
server runs a lot of other things, like 2 web servers, named, squid,
and a whole lot of custom written software, and it pretty much does
everything that both of my other dedicated servers do, so that may be
why it was set so low.
Could this be one of the reasons I see so many bounces in the log?
Would this act like a soft bounce? Besides the process limit what
else should be raised?
smtpd_recipient_restrictions = permit_mynetworks,
reject_unauth_destination, reject_invalid_hostname,
reject_unauth_pipelining, reject_non_fqdn_sender,
reject_unknown_sender_domain, reject_non_fqdn_recipient,
reject_unknown_recipient_domain, reject_rbl_client
bl.spamcop.net, reject_rbl_client cbl.abuseat.org, permit
OK. I suggest dropping cbl.abuseat.org and adding zen.spamhaus.org
(zen includes cbl data).
I was using zen.spamhaus.org, but it seemed to create too many false
pastives. Many emails I was not getting and it was making people mad.
This is when things really started to become a problem, I started
getting duplicate emails, although I can't find anything in the
Postfix log. I just started procmail logging, so I will see if it
shows anything. I think spammers are sending emails that cause this
problem, but I am not sure.
I also like using reject_unknown_reverse_client_hostname to reject
zombies with no rDNS record.
http://www.postfix.org/postconf.
5.html#reject_unknown_reverse_client_hostname
I will look at this. I know from looking at the headers that some
servers that should be able to send email to me will not be able to
if I use this. Is there a way that I can whitelist servers from this?
unknown_local_recipient_reject_code = 550
Good.
Consider a lower smtpd_hard_error_limit so that postfix can
disconnect misbehaving clients sooner. Something between 2..10 is
probably good for most sites.
http://www.postfix.org/postconf.5.html#smtpd_hard_error_limit
This is something that I will definitely put into my config.
Consider using the postfix anvil service to limit how much mail
individual clients can send. Note: anvil is not for traffic
shaping. You may need to exempt a few high volume clients, such as
your forwarders.
http://www.postfix.org/TUNING_README.html#conn_limit
If you have repeat offenders that send lots of spam, firewall
them. You can use fail2ban to automatically temporarily blacklist
clients that exceed a set number of rejects per time period.
http://www.fail2ban.org
Right now, I don't have too many people who relay email through my
server, although it does relay the email for my other servers. I just
don't give anyone new a chance to send spam, because at one time I
had a problem with this.
If system load is a problem, consider running a recent postfix
snapshot with the new postscreen service. The intent of postscreen
is to reject as much spam as possible using as few system resources
as possible. (You'll need to upgrade somewhat frequently to stay
with reasonably current snapshots.)
http://www.postfix.org/POSTSCREEN_README.html
I will look into this. This may be a great solution. The link doesn't
seem to work right now.
I was wondering if using something like policyd would help the
spam problem?
Your time will probably be best spent in identifying the actual
problem and addressing it, rather than just bolting a bunch of
stuff into postfix hoping something will change.
Once you identify the problem as something policyd might help, then
policyd is worth trying.
Right now, the big issue is spam, somewhere some emails are being
duplicated, and the fact that my postfix log looks like a war zone.
On average I don't get a lot of connections, but at times I get what
I would consider a real high number in a very short time (I think
these are an attempt to overload the server). It is not so much that
it is overloaded as I am tired of all the bogus connections and I
really think I need to deal with them better.
Is there a proper way to filter spam? If so, what is it?
If it was easy, no one would get spam. This situation is
complicated since the type of spam and the tolerance for false
positives are local issues. Sounds as if a lot of your spam is
forwarded from accounts on other servers; that's something
SpamAssassin and clamav+sanesecurity sigs can help with.
You can have great success if you can spend time and energy on it;
otherwise just sign up for google apps and gmail.
I spend a lot of time trying to deal with spam. What I have found is
that I need to update my spam filtering often, but still I seem to
need to totally revamp the way that I am dealing with spam. I can't
seem to get away with a lot of false positives, yet I don't want to
deliver the amount of spam that I have been.
I have several websites that I own that are in the top 1,000,000
sites based on traffic according to Alexa and although this server
only hosts the email for like 30 some domains. I seem to get more
than my fair share of spam. Right now, it is still manageable, but
soon I will need a very high end dedicated mail server, if I don't
change something. Personally, I feel my config is wrong and that is
why I am asking some questions.
I was also looking at something else and it looks like Postfix was
built without pcre. Will I be able to use header checks without this?
Sincerely,
Al
