|
Hi, We are running several SpamAssassin servers and are very
pleased with their accuracy, especially after adding most of the custom rules
found at the SARE. I have set up a test qmail server to do a little
benchmarking and ran into something a little odd. No matter what we try to do,
qmail-scanner/clamd/spamassassin take about 6 seconds to process each incoming
message. The test qmail server is configured a new way we are
experimenting with, so the mail server can have high inbound concurrency yet
not overload itself by running out of RAM etc. I have two qmails set up: /var/qmail-in
and /var/qmail-out … qmail-in is accepting 150 concurrencyincoming, and
in its smtproutes file I have it routing everything to the localhost. Qmail-out
is listening on the localhost IP and is what’s actually running
qmail-scanner / SpamAssassin / clamd. So basically qmail-in can accept a ton of incoming SMTP
messages without rejecting any connections… and it queues the messages up
and slowly feeds them to qmail-out, which scans them as fast as it can, then it
delivers the messages. I have played with different “concurrencyremote”
values for qmail-in, to control how many messages it delivers at once to
qmail-out. My tests have shown that if I set concurrencyremote to 1, qmail-out
processes one message every 6 seconds on average. If I set concurrencyremote to
2, it processes 2 messages every 12 seconds. And so on – no matter how I
increase the number of deliveries, it still takes roughly 6 seconds times
however many concurrent deliveries to process the messages. On our normal qmail
servers running just once instance of qmail this is about what we’re
seeing as well – 6 seconds on average per message, or about 600 messages
per hour processed, max. We tried playing with re-nicing qmail-scanner-queue.pl,
spamd, and clamd, but that doesn’t seem to help very much. We are running remote checks since they help improve the
accuracy so much. They are all set up to time out after 5 seconds. I tried
lowering the timeout to 2 or 3 seconds but that didn’t seem to help. We
are using DCC/Pyzor/Razor plus the other built in RBL checks SA does. Any ideas how to speed up SpamAssassin short of pruning the
rules we are using? 1 message every 6 seconds is only 600 messages an hour,
which seems a bit slow. I’m running spamd with these options (using
daemontools): /usr/bin/spamd -m 10 -u spamd -H /home/spamd -x -q –D The SQL checks its doing are to a local MySQL server…
When I look at the spamd logs it looks like spamd is taking most of its time “running
body-text per-line regexp tests”… To be fair, with spamd turned off, qmail-scanner only uses
clamd and it takes 2 or 3 seconds to scan each message, so SpamAssassin is
probably only using up 3-4 of the 6 seconds its taking to process each message. |
