Re: [SAtalk] spam assassin - tru64 unix

2004-01-28 Thread Rick Beebe
Todd Seeleman wrote:
I'm running spamd/spamc v 2.55 on a Compaq DS20 running Tru64 UNIX v
5.1b.  It processes ~ 5,500 pieces of email per day.  Every week or so the
system slows to a point where I must reboot.  I believe I've eliminated all
causes other than the spamd process.  I've throttled it down to spamd -d -m
2 and this has helped so far.  Does anybody have any helpful info on this?
I'm about to start investigating Perl on Tru64. We have two DS20Es in a 
TruCluster running 5.1A. SA is processing around 80,000 messages per 
day. Of course it's not evenly distributed and we've had severe problems 
when it climbs above 22,000 per hour. The load average soars to 40 or 
60, and the box just crawls. If we leave it alone it will eventually 
recover but mail delivery is definitely impacted. Sometimes I just shut 
SA down for a bit to clear out the mail queues. We've done a bunch of 
performance tuning with HP and it helped a little. The load average is 
still floating around 20 though. These boxes, in addition to running SA 
and our MTA also support 1300 simultaneous IMAP connections so the load 
isn't all SA.

Last week, for unrelated reasons, I decided to temporarily put spamd on 
a Linux box. It's a 2.6gHz Zeon box running RedHat 7.3. To my amazement 
the load average on the Tru64 boxes dropped to 3-4 and the load on the 
Linux box never exceeds 2. So, for some reason spamd is _much_ more 
efficient under Linux than under Tru64. My guess is it has something to 
do with the Perl installation but I'm not sure yet.

--
___
   Rick Beebe(203) 785-6416
   Manager, Systems & Network Engineering   FAX: (203) 785-3481
   ITS-Med Production Systems[EMAIL PROTECTED]
   Yale University School of Medicine
   Suite 124, 100 Church Street South   http://its.med.yale.edu
   New Haven, CT 06519
___
---
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk


[SAtalk] SA loads & times

2003-10-28 Thread Rick Beebe
I just updated my production SA environment from 2.41 to 2.60 and am 
running into load issues. I'm running SA on a 2-node Tru64 cluster. Each 
node has 4 gigs of memory and dual 667mHz Alpha CPUs. We process about 
150,000 messages per day but, of course, it isn't an even distribution. 
We can get up to 25,000 messages an hour during the day. We're using 
spamd/spamc with only local tests. I'm struggling with two issues. One 
is the amount of memory spamd is using. I had started with evilrules, 
popcorn, etc but had to take them back out as spamd was sucking up too 
much memory. Without those, it's still considerably larger than 2.4 was. 
The second problem is that spamd is taking longer to process the 
messages. It's doing more, obviously. 2.41 processed most messages in 
less than a second (usually 0.4 - 0.6). 2.60 is taking 1.4 - 20 seconds. 
The problem here is that incoming mail starts to back up. So I try to 
allow more spamd processes to handle it then the machine bogs down. I've 
been playing with -m to try to find an optimal setting.

Is anyone else running SA in a largish environment? I'm looking for any 
and all tuning hints as I really don't want to go back to 2.4. I am 
using a system-wide bayes. I'm not sure how much drag that adds either.

--
___
  Rick Beebe(203) 785-6416
  Manager, Systems & Network Engineering   FAX: (203) 785-3481
  ITS-Med Production Systems[EMAIL PROTECTED]
  Yale University School of Medicine
  Suite 124, 100 Church Street South   http://its.med.yale.edu
  New Haven, CT 06519
___


---
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?   SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk


Re: [SAtalk] Re: SA loads & times

2003-10-29 Thread Rick Beebe
[EMAIL PROTECTED] wrote:

Check for lock files.  There seems to be a 20 second timeout if (in my 
case, it was the whitelist lockfile) gets stale.  Look for minutes/days 
old lockfiles, and clean them out.
I had steady 29+second spamd times, turned off all lookup etc, nothing 
changed.  Snooped in the spamd users dir (I also run spamd on a remote 
machine with the default user 'pop3', so all bayes/whitelist info goes 
into that dir).  Lo and behold a 15 minute old lockfile.  deleted that 
one file and down to < 2 seconds per msg.
dual AMD 1.2Ghz, 1Gb RAM, 3 UW-SCSI 9Gb HDDs RH 9.0.
Thanks. I'll keep an eye on that. In my case I'm only using a global 
directory--there are no per-user files. No AWL and a single global 
bayes. I'll keep an eye out for lock files there.

--
_______
  Rick Beebe(203) 785-6416
  Manager, Systems & Network Engineering   FAX: (203) 785-3481
  ITS-Med Production Systems[EMAIL PROTECTED]
  Yale University School of Medicine
  Suite 124, 100 Church Street South   http://its.med.yale.edu
  New Haven, CT 06519
___


---
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?   SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk


Re: [SAtalk] SA loads & times

2003-11-03 Thread Rick Beebe
Rick Beebe wrote:

I just updated my production SA environment from 2.41 to 2.60 and am 
running into load issues. I'm running SA on a 2-node Tru64 cluster. Each 
node has 4 gigs of memory and dual 667mHz Alpha CPUs. We process about 
150,000 messages per day but, of course, it isn't an even distribution. 
We can get up to 25,000 messages an hour during the day. We're using 
spamd/spamc with only local tests. I'm struggling with two issues. One 
is the amount of memory spamd is using. I had started with evilrules, 
popcorn, etc but had to take them back out as spamd was sucking up too 
much memory. Without those, it's still considerably larger than 2.4 was. 
The second problem is that spamd is taking longer to process the 
messages. It's doing more, obviously. 2.41 processed most messages in 
less than a second (usually 0.4 - 0.6). 2.60 is taking 1.4 - 20 seconds. 
The problem here is that incoming mail starts to back up. So I try to 
allow more spamd processes to handle it then the machine bogs down. I've 
been playing with -m to try to find an optimal setting.
Answering my own question: I discovered that much of this was caused by 
file-locking problems. [EMAIL PROTECTED] put me on the right trail. 
Basically, the copies of spamd running on each node of the cluster were 
sharing a single directory. There were problems with conflicting lock 
files as well as conflicting updates to the bayes database. I fixed it 
by creating separate spamd user home directories for each machine. The 
Bayes databases will eventually get out of sync but since both machines 
are handling about half the spam they should be reasonably close. I am 
still seeing higher load averages and I did under 2.41 but the machine 
isn't grinding to a halt anymore.

--
_______
  Rick Beebe(203) 785-6416
  Manager, Systems & Network Engineering   FAX: (203) 785-3481
  ITS-Med Production Systems[EMAIL PROTECTED]
  Yale University School of Medicine
  Suite 124, 100 Church Street South   http://its.med.yale.edu
  New Haven, CT 06519
___


---
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?   SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk


[SAtalk] "Naughty" test names

2003-06-27 Thread Rick Beebe
I'd like to appeal to the SA collective to change the name of the 
PENIS_ENLARGE tests to something a little more innocuous. Apparently 
some people find it offensive to see it in their email. I've received 
the odd complaint about the test name--usually when it shows up in a 
message that isn't about body part enhancement. I've been ignoring them, 
but apparently there is a larger client rumbling going on and now it's 
gotten to HR who are receiving complaints that it's sexual harrasement. 
All of which is ridiculous, so please don't go there. But I still have 
to deal with it. I asked for this once before and was told to zero out 
the test and put a replacement in my local.cf. I don't want to do that 
because then I lose the collective wisdom of this group. I've had some 
other thoughts on how to fix it, but a fork in the eye is against 
company policy. So I've taken to doing a search and replace changing it 
to BIGGER_BITS. It will be a minor pain to do that every time I upgrade, 
but it's doable. It doesn't seem like it should be a big deal to make 
that change globally, though. Is it? Thanks for your consideration.

--Rick



---
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa0016ave/direct;at.asp_061203_01/01
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk


Re: [spamassassin] Re: [SAtalk] "Naughty" test names

2003-07-03 Thread Rick Beebe
Don Krause wrote:

How can "PENIS" be offensive? It's a medically correct term.

Just offer to turn off SA for those who are so easily offended. I'm sure
they'd prefer PENIS to the content of the actual mail.
Yes, it's medically correct. In all the cases where someone complained, 
however, there was no mention of penis in the email message. They are 
mad at "me" for adding it. The messages were not spam and were not 
marked as spam. They were legitimate email messages which, for whatever 
reason, failed the PENIS_ENLARGE test.

--Rick



---
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa0016ave/direct;at.asp_061203_01/01
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk


Re: [SAtalk] compiling problems on a SGI

2002-07-24 Thread Rick Beebe

Dave Encisco wrote:
> Hi,
> 
> When compiling Mail-SpamAssassin-2.31 on an SGI running 6.5.15 I run
> into the following error:
> 
> cp lib/Mail/SpamAssassin/PerMsgStatus.pm blib/lib/Mail/SpamAssassin/PerMsgStatus.pm
> gcc -fno-strict-aliasing -I/usr/local/include -DLANGUAGE_C  -O spamd/spamc.c 
>\
> -o spamd/spamc -Wl,-woff,84 -L/usr/local/lib -ldb -lm -lc
> In file included from spamd/spamc.c:13:
> /usr/include/sysexits.h:107: warning: `EX_OK' redefined
> /usr/include/unistd.h:43: warning: this is the location of the previous definition
> spamd/spamc.c:50: conflicting types for `in_addr_t'
> /usr/include/netinet/in.h:43: previous declaration of `in_addr_t'
> *** Error code 1 (bu21)
> 
> Has anyone had this problem? Some advice would be greatly appreciated.

I had the same problem with Tru64 Unix. The quick-and-dirty fix is to 
comment out or delete line 50 in spamc.c, the one that says

typedef unsigned long   in_addr_t;

--Rick



---
This sf.net email is sponsored by: Jabber - The world's fastest growing 
real-time communications platform! Don't just IM. Build it in! 
http://www.jabber.com/osdn/xim
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk



Re: [SAtalk] Logging spamassassin...

2002-09-30 Thread Rick Beebe

> I'm currently test spamassassin (spamd/spamc) to see if we should
> implement it sitewide here. Is there a way to log how many mails that
> spamassassin tags as spam and how many that aren't counted as spam. If
> it's possible I also want to log the score so we can have our own little
> high score here :)

I use the attached Perl script which grabs the numbers (including 
highest spam score) from my syslog files. The script accepts data from 
stdin. I use a shell script which basically does "spamcount.pl < 
mail.log" and then mails the output to me. The reason for having a 
separate count for each 'host' is that I'm combining logs from cluster 
members.

--Rick

#!/usr/local/bin/perl -w

# Takes the file from standard input looking for spamd 'clean' lines
# and counts how many were counted and how many were spam

my %total;
my %spam;
my %prob_spam;
my $highest = 0;
my $host;
my ($total, $spam, $prob_spam) = 0;

while (<>) {
next if ! /\s(\w+)\sspamd/;
$host = $1;
next if $host eq '';
if (/(clean message|identified spam)\s+\(([\d\-\.]+)\/([\d\-\.]+)\)/) {
   $total{$host}++;
   $spam{$host}++ if $2 >= $3;
   $prob_spam{$host}++ if $2 >= 5;
   $highest = $2 if $2 > $highest;
}
}

print 

Re: [SAtalk] spamassassin site wide

2002-10-15 Thread Rick Beebe

Theo Van Dinter wrote:
> On Mon, Oct 14, 2002 at 01:06:26PM -0700, Seby Varghese wrote:
> 
>>I have the SpamAssassin installed site wide and would
>>like to know there is a way to configure that if the 
>>user is in white list don't do any other checks. We
>>need this because 80% of the mail handled by our mail 
> 
> There is no current way to do this in SA.  You'd want to configure
> whatever you have running SA to not run SA if given conditions exist. :(

It seems to me that there could be a major performance boost if SA did 
white/blacklist checking first and if the message met either criteria it 
just skipped all the rest of the tests. Is that a bad idea for some reason?

--Rick



---
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk



Re: [SAtalk] spamd uses a lot of CPU

2002-10-23 Thread Rick Beebe
Scott Lambert wrote:

On Wed, Oct 23, 2002 at 10:01:21AM -0400, Chris Santerre wrote:


"Of course, you probably need to use a userprefs database in that case,
rather than the userprefs in each user's home directory."

Aside from moving the process to another system, Are you saying that seting
up the users in a database is a faster process then putting configs in user
directories? I haven't completed my setup for the company. I've just been
testing. I was going to use mailertable for people without custom settings,
and just setup users who wanted custom in regular home dirs. 

I've only got about 100-120 email addresses to handle. So maybe this doesn't
even matter to me.


I have learned to dislike NFS dependencies and user home directories on 
boxes that don't absolutely need them.  I will take a database over those 
options anyday.

The database also gives you one place to look for/change settings.  Most
of my users are lucky to know which mouse button to use at any time.
Editing userprefs files would blow their minds.  Eventually I will put
up a web frontend so they can tweak their own settings in the database
but, until then, it is not a major burden to us to add the entries for
the users who need/want special settings.

We've been having good luck using MySQL for the userprefs. I used the 
php-sa-mysql web front-end with a few minor tweaks.

--Rick



---
This sf.net email is sponsored by: Influence the future 
of Java(TM) technology. Join the Java Community 
Process(SM) (JCP(SM)) program now. 
http://ads.sourceforge.net/cgi-bin/redirect.pl?sunm0002en

___
Spamassassin-talk mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/spamassassin-talk