Hello all,
Got a quick question for all those out there running large setups using
qpsmtpd. We currently have a setup which is load balanced across an several
qpsmtpd servers running a custom linux distro that basically runs the entire
setup in ram for maximum performance. Until lately the setup has been
running absolutely wonderful but with all these spam bots running around
I've started noticing some odd issues and I'm about to start pulling my hair
out. I've tweaked and tweaked the sysctl vars for the network interfaces,
disabled rdns, and also removed almost all of our plugins in an attempt to
keep the machines running with the flood of incoming emails. My setup is as
follows:
Qpsmtpd v0.32 running with forkserver
Lidrectord on the load balancer using masq to backend servers
Below is the exact problem I'm seeing:
Incoming connections start and everything is fine
Load average on the box slowly starts to rise as the connections per second
rises to around 20-30.
Load continues to rise until the max connection is reach or the box become
unresponsive and the load balancer pulls it out.
The odd thing, to me anyways, is that I'm showing 70 - 80% user cpu
utilization and 20% system utilization. Ram is fine and nothing is being
swapped. Using vmstate I'm show ~2000 cs using a 5 second refresh.
Now what's really odd is that occasionally everything goes "normal" and the
load doesn't rise above about a 2 and everything is working fine. During
these times the cps to each box is exactly the same if not higher than it is
during the problem times.
So I guess my question here is, has anyone seen this before or have any
ideas on how to fix it? What performance/tuning settings do you guys have
in place?
Current plugin list:
count_unrecognized_commands
dnsbl
db_get_options - custom
dbwhitelist - custom
check_spamhelo
check_badmailfrom
chck_badrcptto
sender_permitted_from spf_deny 1
denysoft_greylist per_recipient 1
rcpt_mysql - custom
qscanner - custom (think qmail-scanner in a plugin)
Spamassassin
# run the virus checking plugins
virus/trophie - custom
virus/clamdscan
# Log good mail
logging/maillog
# queue mail to postfix
queue/postfix-queue
# Log denied mail
logging/denylog
Any help would be greatly appreciated.
--
Ed McLain
