On Fri, May 06, 2005 at 01:51:13PM +0100, Piers Kittel wrote: > found that after a few hours, I suddenly experience very high load > averages, and extremely slow server performance for everything else, but > if I restart spamassassin, the server works fine again, but it would > start getting high load averages again later on and so on. [...] > Here's a snippet from top when the server is crunching through spamassassin: > > top - 13:07:59 up 51 days, 36 min, 2 users, load average: 7.16, 7.38, 6.28 > Tasks: 116 total, 1 running, 113 sleeping, 0 stopped, 2 zombie > Cpu(s): 1.9% us, 1.9% sy, 0.0% ni, 0.0% id, 96.2% wa, 0.0% hi, 0.0% si > > And after disabling spamassassin, I get: > > top - 13:37:07 up 51 days, 1:06, 2 users, load average: 0.07, 0.04, 0.94 > Tasks: 96 total, 1 running, 95 sleeping, 0 stopped, 0 zombie > Cpu(s): 0.5% us, 0.0% sy, 0.0% ni, 98.0% id, 1.5% wa, 0.0% hi, 0.0% si > > Can I ask if this is normal, or am I doing something wrong somewhere?
Hi Piers, I had the same problem, which is basically that more spam comes in than the server can handle. The main reason for this is not cpu (on _average_ 100% cpu all the time should be enough to handle all spam) but because you run out of memory, the server starts swapping and it becomes too slow to handle the incoming spam. As a result the number of child processes run up till very high numbers (30 till 100 say) and the machine becomes totally unusable. There are several things you can do about this: 1) Reduce the maximum number of child processes that are allowed to run simultaneously. You can do this by passing --max-children 4 to /usr/bin/spamd when starting it. That is, I use 4, you might need more if you have more spam to handle (and the capability to run 4 of them in parallel), which means however: 2) Increase the ammount of memory in the box. You can balance this out with the number of child processes you run thus (see above). My guess is that you need about 32 Mb per child process, but that is a very wild guess, it might be more. The above didn't help me, therefore: 3) Reduce the ammount of spam that spamassassin has to handle! This might seem stupid, but the reason that my server started to lockup every few days turned out to be caused by ONE spam source! Some total idiot had started to send me bursts of spam, all from a single IP-number. By just firewalling that single IP-number I reduced the spam with 90%. Got rid of the burst, and everything worked again. Moreover, you might want to firewall (or reject their mail otherwise before it reaches spamassassin) all of South Korea and all of China -- that will reduce the ammount of spam you receive with about 99% ... So, it is more than worth it. If that is too drastic for you, then try to get the statistics of who is sending you nothing but spam. Likely there are a few B or C classes that ONLY send spam to you and are responsible for over 90% of the spam you get. Finally, you can reduce the spam again DRASTICALLY if you suffer from 'dictionary' attacks: try to find out what the bulk of the spam that you receive is addressed to (that spamassassin is seeing). If the majority of the spam is addressed to non-existant addresses (as was the case in my case) then adding a filter that rejects mail on recipient before it reaches spamassassin again greatly reduces the ammount of spam it has to process. I am doing this by having the following in my /etc/tcpcontrol/smtp.rules: [...] # Use Qmail-Scanner with SpamAssassin on any mail from the rest of the world :allow,RCPTCHECK="/usr/local/bin/rcptcheck",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl" Where /usr/local/bin/rcptcheck is a little program that I wrote myself which simply rejects mails based on the account name it is sent too (Google for RCPTCHECK, assuming you are using qmail here). Well, you get the idea. When my mail server started locking up and I had been resetting it for weeks (like you are doing now) I got real mad, stopped with what I was doing, started to investigate it, and ended up with reducing the amount of spam that spamassassin had to deal with a factor of one thousand, if not more. -- Carlo Wood <[EMAIL PROTECTED]> PS Here is a list of IP-numbers that I firewall to reduce spam with a factor of (more than) 100. CHINANET Shanghai province network: 218.1.0.0/16 222.64.0.0/13 222.72.0.0/15 And South Korea: 59.0.0.0/11 59.150.0.0/16 59.186.0.0/15 60.196.0.0/15 61.4.192.0/19 61.32.0.0/13 61.40.0.0/14 61.72.0.0/13 61.80.0.0/14 61.84.0.0/15 61.96.0.0/12 61.247.128.0/19 61.248.0.0/13 128.134.0.0/16 129.254.0.0/16 134.75.0.0/16 137.68.0.0/16 141.223.0.0/16 143.248.0.0/16 147.6.0.0/16 147.43.0.0/16 150.150.0.0/16 150.183.0.0/16 152.99.0.0/16 152.149.0.0/16 154.10.0.0/16 155.230.0.0/16 156.147.0.0/16 158.44.0.0/16 161.122.0.0/16 163.152.0.0/16 163.180.0.0/16 163.239.0.0/16 164.124.0.0/15 165.132.0.0/15 165.141.0.0/16 165.194.0.0/16 165.213.0.0/16 165.243.0.0/16 165.244.0.0/16 165.246.0.0/16 166.79.0.0/16 166.103.0.0/16 166.104.0.0/16 166.125.0.0/16 168.78.0.0/16 168.115.0.0/16 168.126.0.0/16 168.131.0.0/16 168.154.0.0/16 168.188.0.0/16 168.219.0.0/16 168.248.0.0/15 169.140.0.0/16 192.5.90.0/24 192.100.2.0/24 192.104.15.0/24 192.132.15.0/24 192.132.247.0/24 192.132.248.0/22 192.195.39.0/24 192.195.40.0/24 192.203.138.0/23 192.203.140.0/22 192.203.144.0/23 192.203.146.0/24 192.245.249.0/24 192.245.250.0/23 192.249.16.0/20 202.6.95.0/24 202.14.103.0/24 202.14.165.0/24 202.20.82.0/23 202.20.84.0/23 202.20.86.0/24 202.20.99.0/24 202.20.119.0/24 202.20.128.0/17 202.21.0.0/21 202.30.0.0/15 202.86.8.0/21 202.189.128.0/20 203.81.128.0/19 203.83.128.0/19 203.90.32.0/19 203.100.160.0/19 203.224.0.0/11 210.16.192.0/18 210.80.96.0/19 210.90.0.0/15 210.92.0.0/14 210.96.0.0/11 210.178.0.0/15 210.180.0.0/14 210.204.0.0/14 210.216.0.0/13 211.32.0.0/11 211.104.0.0/13 211.112.0.0/13 211.168.0.0/13 211.176.0.0/12 211.192.0.0/10 218.36.0.0/14 218.48.0.0/13 218.101.128.0/17 218.144.0.0/12 218.232.0.0/13 219.240.0.0/15 219.248.0.0/13 220.64.0.0/11 220.103.0.0/16 220.116.0.0/14 220.120.0.0/13 220.149.0.0/16 221.132.64.0/19 221.133.128.0/18 221.138.0.0/15 221.140.0.0/14 221.144.0.0/12 221.160.0.0/13 221.168.0.0/16 222.96.0.0/12 222.112.0.0/13 222.120.0.0/15 222.122.0.0/16 222.231.0.0/18 222.232.0.0/13
