Hello All,
Justin: SpamAssassin 3.1.0 is a lot better at effective RAM usage than 3.0.0 ... This is because it (a) runs with a smaller number of active children ... keeps one or two servers very busy, using the others for "overflow"
We process 60K messages a day and use most of our 10 spamd processes, this "fix" for memory usage will not help us. The only thing that has worked for us is to limit the number of spammy messages reaching SA, and killing off the spamd processes after a few scans.
Steve
Justin Mason wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
jdow writes:
From: <[EMAIL PROTECTED]>
BTDT, bought the T-shirt. Adding memory will help. Short term solutionThis might be "A GOOD TIME" for someone to create a small exposition
can be adding swap space. Another option can be running SA remotely on another machine (users run spamc -d sa.machine.com)
regarding spamassassin memory usage. I note that it seems to hover
around 56 megabyte to 65 megabyte range even when freshly spawned. I
do not run AWL. I have about 5 megabytes of Bayes data. (I train
lightly of late and only when something new turns up.) I am running
a fair number of SARE rule sets, about 1.3 megabytes worth. How does
this add up to 56 megabytes or more? Does perl take that much space?
Or does its expansion of the rule sets make it so large?
OK, here's a quick wrap-up of what I've observed regarding memory, because this has been one of my priorities for 3.1.0:
- - typical perl 5.8.x spamd is about 25-35megs of memory (per "VSZ") using the default ruleset.
- - this goes up as you add additional rules. ;) With rulesets that contain a lot of complexity, they can easily add 10MB to in-core size, no problem. Relatively small ruleset sizes on-disk can indeed map to a large in-memory footprint, since they're compiled by perl into a speed-optimised format.
- - on a linux server, "top" typically reports a very small amount of memory shared between processes ("SHR"). that's a glitch in linux 2.4.x and 2.6.x kernels; in fact, a lot more RAM is shared, it's just not tracked as such by the kernel any more, although the documentation all states that it is. (annoying.) In fact, about 70% of the memory usage of all the spamd children is shared between them.
- - SpamAssassin 3.1.0 is a lot better at effective RAM usage than 3.0.0, using its new preforking algorithm. This is because it (a) runs with a smaller number of active children, and (b) keeps one or two servers very busy, using the others for "overflow", instead of round-robin serving across all the servers equally (which increases swapping).
- - typically if a spamd process is reported to have over 100 MB of RAM usage, it's indicating a problem with one of the bits of code that uses DB_File -- AWL or Bayes. Extremely large AWL/Bayes db files can cause a spamd process to bloat up massively. If you're seeing this, go check "find" for very large db files...
- - perl 5.6.x uses less RAM than 5.8.x (and is faster ;).
- --j. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Exmh CVS
iD8DBQFCeVJZMJF5cimLx9ARAjHfAJoD5dS3IrxySXbKYJ7MfupgSEbxzACgmG7Z kCUM/hKJ+vVfp7PdD00AKIo= =GLR+ -----END PGP SIGNATURE-----
-- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.308 / Virus Database: 266.11.3 - Release Date: 5/3/2005
