P.A wrote:
Nataraj, thanks for the reply, below is the postconf -n output.
As far as your explanation as to why the other services are slow, pop/imap,
it may be that the TCP stack is under heavy load and might slow down these
connections but the server CPU/MEM are fine and the connections are in est.
state not time_wait.
Off the top of you head do you have any idea what else I can use to eliviate
the issue, sorry for not posting the config.
[r...@pop ~]# postconf -n
alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases
bounce_queue_lifetime = 0
bounce_template_file = /etc/postfix/bounce.cf
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 1
default_destination_concurrency_limit = 15
default_process_limit = 200
fast_flush_domains = $relay_domains
home_mailbox = Maildir/
html_directory = no
inet_interfaces = all
mail_owner = postfix
mail_spool_directory = /var/spool/mail
mailbox_size_limit = 0
mailq_path = /usr/bin/mailq
manpage_directory = /usr/local/man
maximal_queue_lifetime = 1
message_size_limit = 26000000
mydestination = $myhostname, localhost, hash:/etc/postfix/domain-accept
myhostname = pop.cape.com
mynetworks = hash:/etc/postfix/ip-relay
newaliases_path = /usr/bin/newaliases
queue_directory = /var/spool/postfix
readme_directory = no
relay_domains = hash:/etc/postfix/domain-relay
sample_directory = /etc/postfix
sendmail_path = /usr/sbin/sendmail
setgid_group = postdrop
smtp_helo_timeout = 100
smtp_rset_timeout = 22s
smtp_sasl_security_options = noanonymous, nodictionary, noactive
smtpd_banner = $myhostname ESMTP $mail_name **** Networks that SPAM will be
BLOCKED ****
smtpd_client_connection_count_limit = 100
smtpd_error_sleep_time = 0
smtpd_hard_error_limit = 8
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, reject_invalid_hostname,
reject_non_fqdn_hostname
smtpd_recipient_restrictions = regexp:/etc/postfix/recipient_regexp,
permit_sasl_authenticated, reject_non_fqdn_recipient,
reject_unknown_recipient_domain, permit_mynetworks,
reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_sender_restrictions = permit_mynetworks, reject_unauth_pipelining,
reject_non_fqdn_sender, reject_unknown_sender_domain, permit
smtpd_soft_error_limit = 5
smtpd_timeout = 160
strict_rfc821_envelopes = yes
transport_maps = hash:/etc/postfix/transport
unknown_local_recipient_reject_code = 550
unverified_recipient_reject_code = 550
virtual_alias_domains = hash:/etc/postfix/virtdoms
virtual_alias_maps = hash:/etc/postfix/virtmaps
I would take a quick scan through the various smtpd_*_limit and smtpd
time related parameters. I don't have a great memory for these, but I
do something like postconf | grep -i smtpd | grep -i limit to find them
and then look them up in http://www.postfix.org/postconf.5.html
The first ones that I can remember are ones like
smtpd_client_connection_rate_limit smtpd_client_connection_count_limit .
Keep in mind that reducing some limits can make things worse in certain
cases if the machine is being hit harder with failed connection
attempts. I can't be more specific cause it's been a while since I've
played with these. You also need to check your log files, run tcpdump
etc and determine the patterns of spam attacks that are taking you down
and target those initially in your strategy. Are you getting hundreds
of connections from a small number of servers or just lots of different
servers?
Also in your various restriction policies, try to find the least costly
mechanisms for rejecting things as early on as possible. For example,
rejecting messages that:
1) do not have local recipients on your system
2) do not have valid domainnames in the smtp sender, or smtp hello name
3) do not have valid reverse mappings in the dns of the mail server
4) also optionally checking for matching of the reverse and forward dns
5) rbl
Note that, python or perl written policy daemon is usually way more
costly than the builtin postfix checks, so try to weed out as much as
possible with least resource intensive checks. The rbls are useful, but
watchout for too many dns based checks and make sure your dns setup is
strong before relying on it too heavily.
There are lots of options like the ones I list above in the postfix
configuration parameters. If you read through the postconf web page
link above, you'll find ones that are interesting to you.
Unfortunately one of the costs is that some legitimate companies and
ISP's out there have their mailservers misconfigured and can get some
complaints from your users that their mail is not getting through, in
which case I end up both notifying admins about misconfigured servers
and also adding exceptions to my checking, usually through a simple
berkeley db based exception file.
Here are some of the check that I run. Be careful with
smtpd_recipient_restrictions, because anything that returns a permit
will be allowed through and you don't want to allow relaying. So you
want to allow local mail and users that can relay. You'll notice that I
check vpm_recipient_access first, (after checking for authenticated
relay users), so it kicks out any unathenticated senders of mail not
destined for local use. Some options, I have commented out because
they were too agressive and caused too many user complaints.
Changing 400 series responses to 500 series responses will change
temporary failures into permanent rejections, so be careful. You might
debug first with 450 type responses until your sure that you want to
reject things.
smtpd_recipient_restrictions =
permit_sasl_authenticated
permit_mynetworks
check_recipient_access proxy:pgsql:/etc/postfix/vpm_recipient_access
check_sender_access hash:/etc/postfix/smtpd_sender_access
check_recipient_access hash:/etc/postfix/smtpd_recipient_access
reject_rbl_client dul.dnsbl.sorbs.net
# reject_unlisted_recipient
check_policy_service unix:private/vpm-pfpolicy
reject_unauth_destination
smtpd_sender_restrictions =
permit_mynetworks
reject_unknown_sender_domain
reject_unknown_recipient_domain
# reject_unknown_helo_hostname
unknown_address_reject_code = 550
smtpd_client_restrictions =
check_client_access hash:/etc/postfix/smtpd_client_access
reject_unknown_client_hostname
# reject_unknown_reverse_client_hostname
unknown_client_reject_code = 550
smtpd_helo_restrictions =
check_helo_access hash:/etc/postfix/smtpd_helo_names
reject_invalid_helo_hostname
reject_non_fqdn_helo_hostname
# reject_unknown_helo_hostname
# reject_invalid_hostname
# reject_non_fqdn_hostname
# unknown_hostname_reject_code = 550
smtpd_data_restrictions =
reject_unauth_pipelining
Also look for the various error limit parameters and sleep related
paramaters which introduce delays. These of course are more relevant if
your spam attacks send bogus smtp sequences to your server. You can
also use features like rate limiting and some of the scanning and block
features of iptables or of external firewalls and routers.
Also look at the output of 'netstat -s' to get some idea what is going
on with your tcp/ip stack.
A few more additions, greylisting has been helpful for me, but probably
less so now than before the spammers knew how to work around it. I
mentioned earlier, that one admin I talked to was adding 18 second
delays before giving an smtp greeting and they found that extremely
helpful, though some may consider it unfriendly. I'm not sure if
postfix has the capability to do this or not. There are measures that
I've used temporarily to alleviate spam attacks and then turn them off
once I've gotten more long term management in place, so you might have
to be very agressive at first if your server is being taken down by attacks.
Oh, and relative to why it takes the server down, it is related to a
high interrupt rate. I think maybe the powertop utility (intended to
diagnose why laptops may consume excess power, reducing battery runtime)
might show interrupts caused by something like this, but I don't
remember. I'm sure there are other tools. See the cpu usage, wait
states, system states shown by 'top'.
Nataraj
