On Wed, 10 Sep 2003, Jim wrote:
> On Tue, Sep 09, 2003 at 12:18:10AM -0400, Debbie D wrote:
> > 3 times today I noticed the server running very high loads of 4-6 and as
> > high as 10, normally I run 1% or lower usually closer to .3-.6%. Three days
> > in a row I have gotten over load notices.. Tonight I got an out of memory
> > notice and jumped right on to tail the mail log.. The sheer amount of spam
> > was pouring in at amazing rates.. I grabed upwards of 50 domians names and
> > IP's and plopped them into the Reject the following Users/Hosts/Domains ..
> > the loads came down fairly quick after that but this is making me nervous to
> > be frank.. any suggestions??
>
> First, make sure you're using spamc/spamd and not invoking "spamassassin"
> to save yourself some load.
>
> Secondly, set a limit on the max # of active spamd connections you permit.
> See the -m switch to spamd. Try setting it to maybe 5 or 10 and work
> upwards from there.
Another way is to throttle incoming connections with your MTA.
For example, with sendmail you can adjust the Refuse-Load-Average threshold so to not accept new mail when the machine is busy and the ConnectionRateThrottle to limit the maximum number of new connections per second.
Ok,
I've been meaning to get around to writing this recipie for a while, because a lot of people seem to have this problem, so here we go....
Anybody running spamc from procmail with sendmail is vulnerable to having their server overloaded because of uncontrolled concurancy of local deliveries - despite the options mentioned above, sendmail just doesn't have adequate control of local delivery concurancy.
Using the -m option of spamd is only a partial cure and depending on the amount of ram on your system (or lack of) it can actually make things WORSE as it causes sendmail and procmail processes to back up waiting for spamd's be become available which can run the machine out of memory and put it into swap. (At which point all bets are off)
The answer I found on our system is to return a temporary delivery failure to sendmail in cases where you get a flood of messages the system simply can't scan fast enough. Sendmail tries the local delivery again during the next queue run. I do this from a seperate wrapper script before procmail runs, but that is not convienient for most people already using procmail, so I've adapted the idea to a procmail recipie. So here it is:
SPAMCCUR=`ps -A | grep spamc | wc -l` SPAMCMAX=5
:0
*$ -$SPAMCMAX^0
*$ $SPAMCCUR^0
{
EXITCODE=75
:0
/dev/null
}This should go in your /etc/procmail script after your initial variable setting lines such as SHELL=/bin/bash but before any recipies, and before spamc is run.
The basic idea is it checks for the number of spamd processes running, if it's greater than SPAMCMAX then it exits with EX_TEMPFAIL, sendmail requeues the message to try again on the next queue run, rather than leaving a sendmail and procmail process hanging around.
Cavets, comments and recomendations:
* Make sure that your system supports ps -A, redhat and SYSV type systems do, some BSD types may not. If not alter that line to suit.
* A shell process runs to run ps, grep and wc, which naturally has some overhead. In my experience however, the overhead is negligible compared to spamassassin itself, and the benefits outweigh it by far. To speed it up slightly add the full path for ps grep and wc to avoid path searches...
* Only tested with Sendmail/Procmail combination
* Obviously only works with spamc/spamd, since it greps for spamc. If you use spamassassin instead of spamc (why ?) you could change the grep to look for spamassassin
* You probably want your sendmail queue run times quite short (I use 15 minutes) so that if messages do have to be requeued they don't sit on the queue too long.
* Obviously the optimum SPAMCMAX depends on your server. For a small to medium server, 5 to 10, for something pretty grunty (P4) up to about 20-25 maximum. YMMV.
* Using this in combination with the spamd -m option may improve throughput slightly by doing a limited amount of stacking of waiting procmail processes. For example setting spamd's -m option to 10 and setting SPAMCMAX to 15. (Untested)
* Use at your own risk ! Test, test, test.....etc
I'd be interested to get feedback from people if this script works well for them...
Regards, Simon
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Spamassassin-talk mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/spamassassin-talk
