On 6/9/2010 7:51 PM, Spiro Harvey wrote: > I maintain a mail cluster that gets about 70,000 messages a day per > node. > > I'm just wondering if it's possible to decrease the scan times. In the > TOTALS section AvgTm is the average "scantime" in the spamassassin log > file: > > (Delivered are messages that SA scores under 5, Spamboxed are scored > 5+, but under 10, and Rejected are 10+) > > > > # ./knl-spam-stats.awk /var/log/spamassassin.* > > TOTALS > AvgTm AvgThruput > # Msgs %/Total (sec) (bytes/sec) > ~~~~~~ ~~~~~~~~~ ~~~~~ ~~~~~~~~~~~ > Delivered 176086 ( 17.80%) 15.22 2208.40 > Spamboxed 51194 ( 5.17%) 19.92 550.14 > Rejected 762189 ( 77.03%) 19.30 537.56 > > Total 989469 messages processed > > (70676/day; 2944.85/hr; 49.08/min; 0.82/sec) > > BLACKLIST HITS > Blacklist Msgs %/Total %/Spam Avg Score > ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~ ~~~~~~~~ ~~~~~~~~~ > Spamhaus SBL 2087 ( 0.21%) ( 0.26%) 16.15 > Spamhaus PBL 569825 ( 57.59%) ( 70.06%) 21.99 > Spamhaus XBL 497403 ( 50.27%) ( 61.15%) 21.98 > SBL URI 187292 ( 18.93%) ( 23.03%) 26.05 > NJABL 3544 ( 0.36%) ( 0.44%) 23.65 > SORBS 387539 ( 39.17%) ( 47.65%) 22.32 > Spamcop 513748 ( 51.92%) ( 63.16%) 22.48 > SURBL URI 360620 ( 36.45%) ( 44.34%) 27.25 > RFC Ignorant 29295 ( 2.96%) ( 3.60%) 20.68 > > CUSTOM RULE HITS > Custom Rule Msgs %/Total Avg Score > ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~ ~~~~~~~~~ > MIME/JPG 84 ( 0.01%) 15.45 > ZIP file 741 ( 0.07%) 18.44 > > > Yet, on another mail cluster that only gets 4-5000 messages a day per > node, the average scantimes are 4-5 seconds. Both have the same custom > rules, so any slowness in processing regexes should be noticable on > both systems. > > In the first case, we have started rsyncing Spamhaus' blacklists in the > hopes that it would increase scantimes by decreasing DNS lookup times. > It hasn't really made too much difference, but my main concern is that > the messages seem to be taking so long regardless. > > The boxes are running Sendmail 8.14 + ClamAV 0.96 + SA 3.3.1 + Razor > > > SPAMDOPTIONS="-d -x -c -m50 -H -s local2 /home/spamd -u spamd > --min-children=10 --min-spare=10" > > Core 2 Duo @2.93GHz, 4GB RAM. Load averages typically sit at 5-7 during > the day. > > Any advice on how I can tune the scantimes? > > > These settings: -m 50 --min-children=10 --min-spare=10
seem a bit high for a box with only 4GB of ram... Is the box suffering from severe swap usage, and grinding itself to a halt when all 50 are up and running? (try running "free", what does it say?) I might suggest something more like 10-20 as a max children with 4gb of ram: -m 10 --min-children 5 --min-spare=1 -m 20 --min-children 10 --min-spare=2 Adding more children helps, but only if you have enough ram to fit them all. Once you run out of ram, performance suffers severely. Or, as the manpage for -m says: Note that if you run too many servers for the amount of free RAM available, you run the danger of hurting performance by causing a high swap load as server processes are swapped in and out continually.
