Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-22 Thread Ben Johnson
On 4/20/2013 3:20 PM, Benny Pedersen wrote: > Ben Johnson skrev den 2013-04-20 05:02: > >> Yes, I believe that me and the system always execute SA commands as the >> "amavis" user. When I was using the SQL setup, I had the following in >> local.cf: >> >> bayes_path /var/lib/amavis/.spamassassin/

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-20 Thread Benny Pedersen
Ben Johnson skrev den 2013-04-20 19:01: Welp, that'll do it! How those four files were set to root:root ownership is beyond me, that means that root have doing some testing :) later amavisd cant write, you should change to amavis user before testing su amavis -c cmd foo but that was cert

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-20 Thread Benny Pedersen
Ben Johnson skrev den 2013-04-20 05:02: Yes, I believe that me and the system always execute SA commands as the "amavis" user. When I was using the SQL setup, I had the following in local.cf: bayes_path /var/lib/amavis/.spamassassin/bayes is amavis have homedir in /var/lib/ ? in gentoo its

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-20 Thread Benny Pedersen
Ben Johnson skrev den 2013-04-20 04:40: By "feed it a few thousand NEW spams", do you mean to scrap the training corpora that I've hand-sorted in favor of starting over? Or do you mean to clear the database and re-run the training script against the corpora? ls /path/to/maildir/spam >/tmp/sp

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-20 Thread Ben Johnson
So, the problem seems not to be SQL-specific, as it occurs with SQL or flat-file DB. Upon following Benny Pedersen's advice (to move SA configuration directives from /etc/spamassassin/local.cf to /var/lib/amavis/.spamassassin/user_prefs), I noticed something unusual: $ ls -lah /var/lib/amavis/.sp

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Ben Johnson
Apologies for the rapid-fire here folks, but I wanted to correct something. I had these backwards: >> Yes, I believe that me and the system always execute SA commands as the >> "amavis" user. When I was using the SQL setup, I had the following in >> local.cf: >> >> bayes_path /var/lib/amavis/.sp

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Ben Johnson
On 4/19/2013 1:54 PM, Benny Pedersen wrote: > Ben Johnson skrev den 2013-04-19 18:02: > >> Still stumped here... > > for amavisd-new, put spamassassin sql setup into user_prefs file for the > user amavisd-new runs as might be working better then have insecure sql > settings in /etc/mail/spamass

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Ben Johnson
On 4/19/2013 12:12 PM, Axb wrote: > On 04/19/2013 06:02 PM, Ben Johnson wrote: > >> Still stumped here... > > do a bayes sa-learn --backup > > switch to file based in SDBM format (which is fast) > > do a > > sa-learn --restore > > feed it a few thousand NEW spams > > see what happens > >

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Benny Pedersen
Ben Johnson skrev den 2013-04-19 18:02: Still stumped here... for amavisd-new, put spamassassin sql setup into user_prefs file for the user amavisd-new runs as might be working better then have insecure sql settings in /etc/mail/spamassassin :) i dont know if this is really that you have a

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Benny Pedersen
John Hardin skrev den 2013-04-18 04:15: ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; unicode is overkill since bayes is just ascii it will if unicode is used create bigger db, that will slow down more then ascii Please check the SpamAssassin bugzilla to see if this situation is al

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Axb
On 04/19/2013 06:02 PM, Ben Johnson wrote: Still stumped here... do a bayes sa-learn --backup switch to file based in SDBM format (which is fast) do a sa-learn --restore feed it a few thousand NEW spams see what happens

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Ben Johnson
On 4/19/2013 11:42 AM, Alex wrote: > Hi, > >> Is this normal? If so, what is the explanation for this behavior? I have > > marked dozens of nearly-identical messages with the subject > "Garden hose > expands up to three times its length" as SPAM (over the course of >

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Alex
Hi, > Is this normal? If so, what is the explanation for this behavior? I have > marked dozens of nearly-identical messages with the subject "Garden hose >> expands up to three times its length" as SPAM (over the course of >> several weeks) as SPAM, and yet SA reports "not enough usable tokens >>

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Alex
Hi, > Might anyone be in a position to offer an authoritative response to > these questions? > > I continue to see messages that are very similar to dozens of messages > that have been marked as SPAM slipping through with *no Bayes scoring* > (this is *after* fixing the SQL syntax error issue): >

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-19 Thread Ben Johnson
On 4/18/2013 12:18 PM, Ben Johnson wrote: > > My concern now is that I am on 3.3.1, with little control over upgrades. > I have read all three bug reports in their entirety and Bug 6624 seems > to be a very legitimate concern. To quote Mark in the bug description: > >> The effect of the bug wit

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-18 Thread Alex
Hi, > Curious: what are your reasons for using Bayes in SQL? > > Are you sharing the DB among several machines? Or is this a single > > box/global bayes setup? > > > > > > Not yet, but that is the ultimate plan (to share the DB across multiple > servers). Also, I like the idea that the Bayes DB

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-18 Thread Ben Johnson
On 4/18/2013 12:26 PM, Axb wrote: > On 04/18/2013 06:18 PM, Ben Johnson wrote: >> I have done some searching-around on the string "cannot use bayes on >> this message; not enough usable tokens found" and have not found >> anything authoritative regarding what this message might mean and >> whethe

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-18 Thread Axb
On 04/18/2013 06:18 PM, Ben Johnson wrote: I have done some searching-around on the string "cannot use bayes on this message; not enough usable tokens found" and have not found anything authoritative regarding what this message might mean and whether or not it can be ignored or if it is symptomat

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-18 Thread Ben Johnson
On 4/17/2013 10:15 PM, John Hardin wrote: > On Wed, 17 Apr 2013, Ben Johnson wrote: > >> The first post on that page was the key. In particular, adding the >> following to each MySQL "CREATE TABLE" statement: >> >> ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; > > Please check the SpamAs

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread Ben Johnson
On 4/17/2013 5:39 PM, Tom Hendrikx wrote: > On 17-04-13 21:40, Ben Johnson wrote: >> Ideally, using the above directives will tell us whether we're >> experiencing timeouts, or these spam messages are simply not in the >> Pyzor or Razor2 databases. >> >> Off the top of your head, do you happen to

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread John Hardin
On Wed, 17 Apr 2013, Ben Johnson wrote: The first post on that page was the key. In particular, adding the following to each MySQL "CREATE TABLE" statement: ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; Please check the SpamAssassin bugzilla to see if this situation is already mention

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread Ben Johnson
On 4/17/2013 6:47 PM, Ben Johnson wrote: > > > On 4/17/2013 5:05 PM, Kris Deugau wrote: >> Ben Johnson wrote: >>> Is there anything else that would cause Bayes tests not be performed? I >>> ask because other types of tests are disabled automatically under >>> certain circumstances (e.g., networ

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread Ben Johnson
On 4/17/2013 5:05 PM, Kris Deugau wrote: > Ben Johnson wrote: >> Is there anything else that would cause Bayes tests not be performed? I >> ask because other types of tests are disabled automatically under >> certain circumstances (e.g., network tests), and I'm wondering if there >> is some obscu

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread Tom Hendrikx
On 17-04-13 21:40, Ben Johnson wrote: > Ideally, using the above directives will tell us whether we're > experiencing timeouts, or these spam messages are simply not in the > Pyzor or Razor2 databases. > > Off the top of your head, do you happen to know what will happen if one > or both of the Pyz

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread Kris Deugau
Ben Johnson wrote: > Is there anything else that would cause Bayes tests not be performed? I > ask because other types of tests are disabled automatically under > certain circumstances (e.g., network tests), and I'm wondering if there > is some obscure combination of factors that causes Bayes tests

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-17 Thread Ben Johnson
Daniel, thanks for the quick reply. I'll reply inline, below. On 4/16/2013 5:01 PM, Daniel McDonald wrote: > > > > On 4/16/13 2:59 PM, "Ben Johnson" wrote: > >> Are there any normal circumstances under which Bayes tests are not run? > Yes, if USE_BAYES = 0 is included in the local.cf file. I

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-16 Thread Daniel McDonald
On 4/16/13 2:59 PM, "Ben Johnson" wrote: >Are there any normal circumstances under which Bayes tests are not run? Yes, if USE_BAYES = 0 is included in the local.cf file. > > If not, are there circumstances under which Bayes tests are run but > their results are not included in the message he

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-04-16 Thread Ben Johnson
Apologies for resurrecting the thread, but I never did receive a response to this particular aspect of the problem (asked on Jan 18, 2013, 8:51 AM). This is probably because I replied to my own post before anyone else did, and changed the subject slightly. We are being hammered pretty hard with sp

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-06 Thread John Hardin
On Wed, 6 Feb 2013, Ben Johnson wrote: On 2/1/2013 7:58 PM, John Hardin wrote: That latter brings up another concern for the vetted-corpora model: if a message is *removed* from a training corpora mailbox rather than reclassified, you'd have to wipe and retrain your database from scratch to rem

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-06 Thread Ben Johnson
On 2/1/2013 7:58 PM, John Hardin wrote: > On Sat, 2 Feb 2013, RW wrote: > >> ALLOWING APPENDS >>By appends we mean the case of mail moving when the source folder is >>unknown, e.g. when you move from some other account or with tools >>like offlineimap. You should be careful with allo

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-06 Thread Ben Johnson
On 2/1/2013 12:00 PM, John Hardin wrote: > On Fri, 1 Feb 2013, Ben Johnson wrote: > >> John, thanks for pointing-out the problems associated with re-sending >> the messages via sendmail. >> >> I threw a line out to the Dovecot users group and learned how to move >> messages without going through

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-01 Thread John Hardin
On Sat, 2 Feb 2013, RW wrote: ALLOWING APPENDS By appends we mean the case of mail moving when the source folder is unknown, e.g. when you move from some other account or with tools like offlineimap. You should be careful with allowing APPENDs to SPAM folders. The reason for possibly

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-01 Thread RW
On Fri, 1 Feb 2013 09:00:48 -0800 (PST) John Hardin wrote: > On Fri, 1 Feb 2013, Ben Johnson wrote: > > > John, thanks for pointing-out the problems associated with > > re-sending the messages via sendmail. > > > > I threw a line out to the Dovecot users group and learned how to > > move messages

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-01 Thread John Hardin
On Fri, 1 Feb 2013, Ben Johnson wrote: John, thanks for pointing-out the problems associated with re-sending the messages via sendmail. I threw a line out to the Dovecot users group and learned how to move messages without going through the MTA. Dovecot has a utility executable, "deliver", whic

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-02-01 Thread Ben Johnson
On 1/31/2013 5:50 PM, RW wrote: > On Thu, 31 Jan 2013 12:12:15 -0800 (PST) > John Hardin wrote: > >> On Thu, 31 Jan 2013, Ben Johnson wrote: >> > >>> So, I finally got around to tackling this change. >>> >>> With a couple of simple modifications, I was able to achieve the >>> desired result wit

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-31 Thread RW
On Thu, 31 Jan 2013 12:12:15 -0800 (PST) John Hardin wrote: > On Thu, 31 Jan 2013, Ben Johnson wrote: > > > So, I finally got around to tackling this change. > > > > With a couple of simple modifications, I was able to achieve the > > desired result with the Dovecot Antispam plug-in. > > > > Bas

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-31 Thread John Hardin
On Thu, 31 Jan 2013, Ben Johnson wrote: On 1/15/2013 5:22 PM, John Hardin wrote: Yes, users are allowed to train Bayes, via Dovecot's Antispam plug-in. They do so unsupervised. Why this could be a problem is obvious. And no, I don't retain their submissions. I probably should. I wonder if I c

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-31 Thread Ben Johnson
On 1/15/2013 5:22 PM, John Hardin wrote: Yes, users are allowed to train Bayes, via Dovecot's Antispam plug-in. They do so unsupervised. Why this could be a problem is obvious. And no, I don't retain their submissions. I probably should. I wonder if I can make a few sligh

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-18 Thread Ben Johnson
So, I've been keeping an eye on things again today. Overall, things look pretty good, and most spam is being blocked outright at the MTA and scored appropriately in SA if not. I've been inspecting the X-Spam-Status headers for the handful of messages that do slip through and noticed that most of

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Ben Johnson
On 1/16/2013 2:22 PM, Bowie Bailey wrote: > On 1/16/2013 1:18 PM, Ben Johnson wrote: >> >> On 1/16/2013 11:00 AM, John Hardin wrote: >>> On Wed, 16 Jan 2013, Ben Johnson wrote: >>> Is it possible that the training I've been doing over the last week or so wasn't *effective* until recentl

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Alex
Hi, >>> smtpd_recipient_restrictions = >>> reject_rbl_client bl.spamcop.net, >>> reject_rbl_client list.dsbl.org, >>> reject_rbl_client sbl-xbl.spamhaus.org, >>> reject_rbl_client cbl.abuseat.org, >>> reject_rbl_client dul.dnsbl.sorbs.net, >> >> Several of those are combined in

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Bowie Bailey
On 1/16/2013 1:18 PM, Ben Johnson wrote: On 1/16/2013 11:00 AM, John Hardin wrote: On Wed, 16 Jan 2013, Ben Johnson wrote: Is it possible that the training I've been doing over the last week or so wasn't *effective* until recently, say, after restarting some component of the mail stack? My un

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread John Hardin
On Wed, 16 Jan 2013, Ben Johnson wrote: On 1/16/2013 11:00 AM, John Hardin wrote: That's odd. That suggests you SA wasn't looking up those DNSBLs, or they would have contributed to the score. Check your trusted networks setting. One difference between SMTP-time and SA-time DNSBL checks is tha

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Ben Johnson
On 1/16/2013 11:00 AM, John Hardin wrote: > On Wed, 16 Jan 2013, Ben Johnson wrote: > >> On 1/15/2013 5:22 PM, John Hardin wrote: >>> On Tue, 15 Jan 2013, Ben Johnson wrote: Wow! Adding several more reject_rbl_client entries to the smtpd_recipient_restrictions directive in the Pos

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Noel
On 1/16/2013 9:49 AM, Ben Johnson wrote: > smtpd_recipient_restrictions = > reject_rbl_client bl.spamcop.net, spamcop has a reputation of being somewhat aggressive on blocking, and their website recommends using it in a scoring system (eg. SpamAssassin) rather than for outright blocking. Th

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Bowie Bailey
On 1/16/2013 10:49 AM, Ben Johnson wrote: On 1/15/2013 5:22 PM, John Hardin wrote: On Tue, 15 Jan 2013, Ben Johnson wrote: Wow! Adding several more reject_rbl_client entries to the smtpd_recipient_restrictions directive in the Postfix configuration seems to be having a tremendous impact. The

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread John Hardin
On Wed, 16 Jan 2013, Ben Johnson wrote: On 1/15/2013 5:22 PM, John Hardin wrote: On Tue, 15 Jan 2013, Ben Johnson wrote: Wow! Adding several more reject_rbl_client entries to the smtpd_recipient_restrictions directive in the Postfix configuration seems to be having a tremendous impact. The am

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Ben Johnson
On 1/16/2013 2:02 AM, Tom Hendrikx wrote: > On 1/15/13 5:26 PM, Ben Johnson wrote: > >> >> In postfix's main.cf: >> > >> >> Hmm, very interesting. No, I have no greylisting in place as yet, and >> no, my userbase doesn't demand immediate delivery. I will look into >> greylisting further. > > I

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-16 Thread Ben Johnson
On 1/15/2013 5:22 PM, John Hardin wrote: > On Tue, 15 Jan 2013, Ben Johnson wrote: > >> >> >> On 1/15/2013 1:55 PM, John Hardin wrote: >>> On Tue, 15 Jan 2013, Ben Johnson wrote: >>> On 1/14/2013 8:16 PM, John Hardin wrote: > On Mon, 14 Jan 2013, Ben Johnson wrote: > > Question:

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Tom Hendrikx
On 1/15/13 5:26 PM, Ben Johnson wrote: > > In postfix's main.cf: > > > Hmm, very interesting. No, I have no greylisting in place as yet, and > no, my userbase doesn't demand immediate delivery. I will look into > greylisting further. If you're running postfix, consider using postscreen. It's a

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread jdow
On 2013/01/15 17:23, John Hardin wrote: On Tue, 15 Jan 2013, jdow wrote: On 2013/01/15 08:26, Ben Johnson wrote: Based on my responses, what's the next move? Backup the Bayes DB, wipe it, and feed my corpus through the ol' chipper? (Sure to infuriate BUT - read the WHOLE note.) Are you

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread John Hardin
On Tue, 15 Jan 2013, jdow wrote: On 2013/01/15 08:26, Ben Johnson wrote: Based on my responses, what's the next move? Backup the Bayes DB, wipe it, and feed my corpus through the ol' chipper? (Sure to infuriate BUT - read the WHOLE note.) Are you sure your Bayes database is well trained?

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread jdow
On 2013/01/15 08:26, Ben Johnson wrote: Based on my responses, what's the next move? Backup the Bayes DB, wipe it, and feed my corpus through the ol' chipper? (Sure to infuriate BUT - read the WHOLE note.) Are you sure your Bayes database is well trained? But let's change that to, "Is the Bay

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread jdow
On 2013/01/15 07:27, Ben Johnson wrote: On 1/14/2013 7:48 PM, Noel wrote: On 1/14/2013 2:59 PM, Ben Johnson wrote: jdow, Noel, and John, I can't thank you enough for your very thorough responses. Your time is valuable and I sincerely appreciate your willingness to help. Glad it was even mar

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread John Hardin
On Tue, 15 Jan 2013, Ben Johnson wrote: On 1/15/2013 1:55 PM, John Hardin wrote: On Tue, 15 Jan 2013, Ben Johnson wrote: On 1/14/2013 8:16 PM, John Hardin wrote: On Mon, 14 Jan 2013, Ben Johnson wrote: Question: do you have any SMTP-time hard-reject DNSBL tests in place? Or are they all p

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Ben Johnson
On 1/15/2013 4:39 PM, Bowie Bailey wrote: > On 1/15/2013 4:27 PM, Ben Johnson wrote: >> On 1/15/2013 4:05 PM, Bowie Bailey wrote: >>> On 1/15/2013 3:47 PM, Ben Johnson wrote: One final question on this subject (sorry...). Is there value in training Bayes on messages that SA classif

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Bowie Bailey
On 1/15/2013 4:27 PM, Ben Johnson wrote: On 1/15/2013 4:05 PM, Bowie Bailey wrote: On 1/15/2013 3:47 PM, Ben Johnson wrote: One final question on this subject (sorry...). Is there value in training Bayes on messages that SA classified as spam *due to other test scores*? In other words, if a me

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Ben Johnson
On 1/15/2013 4:05 PM, Bowie Bailey wrote: > On 1/15/2013 3:47 PM, Ben Johnson wrote: >> One final question on this subject (sorry...). >> >> Is there value in training Bayes on messages that SA classified as spam >> *due to other test scores*? In other words, if a message is classified >> as SPAM

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Bowie Bailey
On 1/15/2013 3:47 PM, Ben Johnson wrote: One final question on this subject (sorry...). Is there value in training Bayes on messages that SA classified as spam *due to other test scores*? In other words, if a message is classified as SPAM due to a block-list test, but the message is new enough f

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Ben Johnson
One final question on this subject (sorry...). Is there value in training Bayes on messages that SA classified as spam *due to other test scores*? In other words, if a message is classified as SPAM due to a block-list test, but the message is new enough for Bayes to assign a zero score, should tha

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Ben Johnson
On 1/15/2013 1:55 PM, John Hardin wrote: > On Tue, 15 Jan 2013, Ben Johnson wrote: > >> On 1/14/2013 8:16 PM, John Hardin wrote: >>> On Mon, 14 Jan 2013, Ben Johnson wrote: >>> >>> Question: do you have any SMTP-time hard-reject DNSBL tests in place? Or >>> are they all performed by SA? >> >> In

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread John Hardin
On Tue, 15 Jan 2013, Ben Johnson wrote: On 1/14/2013 8:16 PM, John Hardin wrote: On Mon, 14 Jan 2013, Ben Johnson wrote: Question: do you have any SMTP-time hard-reject DNSBL tests in place? Or are they all performed by SA? In postfix's main.cf: smtpd_recipient_restrictions = permit_mynetwo

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Ben Johnson
On 1/14/2013 8:16 PM, John Hardin wrote: > On Mon, 14 Jan 2013, Ben Johnson wrote: > >> I understand that snowshoe spam may not hit any net tests. I guess my >> confusion is around what, exactly, classifies spam as "snowshoe". > > http://www.spamhaus.org/faq/section/Glossary > > Basically, a

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-15 Thread Ben Johnson
On 1/14/2013 7:48 PM, Noel wrote: > On 1/14/2013 2:59 PM, Ben Johnson wrote: > >> I understand that snowshoe spam may not hit any net tests. I guess my >> confusion is around what, exactly, classifies spam as "snowshoe". > > Snowshoe spam - spreading a spam run across a large number of IPs so >

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread John Hardin
On Mon, 14 Jan 2013, Ben Johnson wrote: I understand that snowshoe spam may not hit any net tests. I guess my confusion is around what, exactly, classifies spam as "snowshoe". http://www.spamhaus.org/faq/section/Glossary Basically, a large number of spambots sending the message so that no o

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread Noel
On 1/14/2013 2:59 PM, Ben Johnson wrote: > I understand that snowshoe spam may not hit any net tests. I guess my > confusion is around what, exactly, classifies spam as "snowshoe". Snowshoe spam - spreading a spam run across a large number of IPs so no single IP is sending a large volume. Typica

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread jdow
On 2013/01/14 12:59, Ben Johnson wrote: On 1/14/2013 2:49 PM, RW wrote: On Mon, 14 Jan 2013 13:24:55 -0500 Ben Johnson wrote: A clear pattern has emerged: the X-Spam-Status headers for very obviously spammy messages never contain evidence that network tests contributed to their SA scores.

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread jdow
On 2013/01/14 10:24, Ben Johnson wrote: On 1/11/2013 4:27 PM, Ben Johnson wrote: I enabled Amavis's SA debugging mode on the server in question and was able to extract the debug output for two messages that seem like they should definitely be classified as spam. Message #1: http://pastebin.co

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread Ben Johnson
On 1/14/2013 2:49 PM, RW wrote: > On Mon, 14 Jan 2013 13:24:55 -0500 > Ben Johnson wrote: > > >> A clear pattern has emerged: the X-Spam-Status headers for very >> obviously spammy messages never contain evidence that network tests >> contributed to their SA scores. >> >> Ultimately, I need to

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread RW
On Mon, 14 Jan 2013 13:24:55 -0500 Ben Johnson wrote: > A clear pattern has emerged: the X-Spam-Status headers for very > obviously spammy messages never contain evidence that network tests > contributed to their SA scores. > > Ultimately, I need to know whether: > > a.) Network tests are not b

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-14 Thread Ben Johnson
On 1/11/2013 4:27 PM, Ben Johnson wrote: > I enabled Amavis's SA debugging mode on the server in question and was > able to extract the debug output for two messages that seem like they > should definitely be classified as spam. > > Message #1: http://pastebin.com/xLMikNJH > > Message #2: http:

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-11 Thread Ben Johnson
On 1/10/2013 3:13 PM, Tom Hendrikx wrote: > On 10-01-13 19:55, Ben Johnson wrote: >> >> >> On 1/10/2013 1:06 PM, RW wrote: >>> On Thu, 10 Jan 2013 12:48:07 -0500 >>> Ben Johnson wrote: pon further consideration, this behavior makes perfect sense if the mailbox user has moved the message

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-11 Thread Ben Johnson
On 1/10/2013 4:12 PM, John Hardin wrote: > On Thu, 10 Jan 2013, Ben Johnson wrote: > >> So, at this point, I'm struggling to understand how the following >> happened. >> >> Over the course of 15 minutes, I received the same exact message four >> times. Each time, the message was sent to the same

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-11 Thread RW
On Thu, 10 Jan 2013 13:55:58 -0500 Ben Johnson wrote: > So, at this point, I'm struggling to understand how the following > happened. > > Over the course of 15 minutes, I received the same exact message four > times. Each time, the message was sent to the same recipient mailbox. > The "From" and

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread John Hardin
On Thu, 10 Jan 2013, Ben Johnson wrote: So, at this point, I'm struggling to understand how the following happened. Over the course of 15 minutes, I received the same exact message four times. Each time, the message was sent to the same recipient mailbox. The "From" and "Return-Path" headers ch

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread Tom Hendrikx
On 10-01-13 19:55, Ben Johnson wrote: > > > On 1/10/2013 1:06 PM, RW wrote: >> On Thu, 10 Jan 2013 12:48:07 -0500 >> Ben Johnson wrote: >>> pon further consideration, this behavior makes perfect sense if the >>> mailbox user has moved the message from Inbox to Junk between scans; >>> Dovecot's An

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread Ben Johnson
On 1/10/2013 1:06 PM, RW wrote: > On Thu, 10 Jan 2013 12:48:07 -0500 > Ben Johnson wrote: >> pon further consideration, this behavior makes perfect sense if the >> mailbox user has moved the message from Inbox to Junk between scans; >> Dovecot's Antispam filter is in use on this server. This acti

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread RW
On Thu, 10 Jan 2013 12:48:07 -0500 Ben Johnson wrote: > pon further consideration, this behavior makes perfect sense if the > mailbox user has moved the message from Inbox to Junk between scans; > Dovecot's Antispam filter is in use on this server. This action would > cause the message tokens to be

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread Ben Johnson
>> > > RW, > > I understand that, but that doesn't explain why if I retest a given > message by calling SpamAssassin directly, and I *disable network tests*, > the score is sometimes *higher* than when the message was scanned > initially with AMaViS. > > When

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread Ben Johnson
been done on this box. > > As has already been said, the score from network tests is commonly a > lot higher on retesting because of all the reporting that happened > in-between. > RW, I understand that, but that doesn't explain why if I retest a given message by calling S

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread RW
On Thu, 10 Jan 2013 11:43:44 -0500 Ben Johnson wrote: > This observation begs the question: why are network tests being > performed for some messages but not others? To my knowledge, no > white/gray/black listing has been done on this box. As has already been said, the score from network tests

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-10 Thread Ben Johnson
On 1/9/2013 9:13 PM, John Hardin wrote: > On Wed, 9 Jan 2013, Ben Johnson wrote: > >> On 1/9/2013 7:36 PM, wolfgang wrote: >>> RCVD_IN_BRBL_LASTEXT,RCVD_IN_CSS,RCVD_IN_PSBL,RCVD_IN_XBL,URIBL_DBL_S PAM, URIBL_JP_SURBL autolearn=disabled version=3.3.1 >>> >>> I am not familiar with amavi

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread Ned Slider
On 10/01/13 00:03, Ben Johnson wrote: On 1/9/2013 5:36 PM, RW wrote: This is not better, it indicates that SA didn't recognise it as an email, not that it recognised it as a spam. Whatever /tmp/msg.txt was it wasn't a properly formatted email. Thanks for the quick replies, Marius and RW.

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread John Hardin
On Wed, 9 Jan 2013, Ben Johnson wrote: On 1/9/2013 7:36 PM, wolfgang wrote: RCVD_IN_BRBL_LASTEXT,RCVD_IN_CSS,RCVD_IN_PSBL,RCVD_IN_XBL,URIBL_DBL_S PAM, URIBL_JP_SURBL autolearn=disabled version=3.3.1 I am not familiar with amavis, but I know that it calls spamassassin in a special way, depen

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread Ben Johnson
On 1/9/2013 7:36 PM, wolfgang wrote: > On 2013-01-10 01:03, Ben Johnson wrote: > >> I see; I saved the email message out of Thunderbird (with View -> >> Headers -> All), as a plain text file. Apparently, that process >> butchers the original message. > > In Thunderbird, rather use File > Save a

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread wolfgang
On 2013-01-10 01:03, Ben Johnson wrote: > I see; I saved the email message out of Thunderbird (with View -> > Headers -> All), as a plain text file. Apparently, that process > butchers the original message. In Thunderbird, rather use File > Save as to save the entire message. > RCVD_IN_BRBL_LAST

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread Ben Johnson
On 1/9/2013 5:36 PM, RW wrote: > On Wed, 09 Jan 2013 17:14:05 -0500 > Ben Johnson wrote: > >> About five months ago, I experienced a problem that I *thought* I had >> resolved, but I am observing similar behavior after retraining the >> Bayes database. While the symptoms are similar, the root ca

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread RW
On Wed, 09 Jan 2013 17:14:05 -0500 Ben Johnson wrote: > About five months ago, I experienced a problem that I *thought* I had > resolved, but I am observing similar behavior after retraining the > Bayes database. While the symptoms are similar, the root cause seems > to be different (thankfully).

Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread Marius Gavrilescu
On Wed, Jan 09, 2013 at 05:14:05PM -0500, Ben Johnson wrote: > Content analysis details: (7.5 points, 5.0 required) > > pts rule name description > -- > -- > -0.0 NO_RELAYS Informational: message w

Calling spamassassin directly yields very different results than calling spamassassin via amavis-new

2013-01-09 Thread Ben Johnson
1382400 0 non-token data: last expire atime delta 0.000 0 3191 0 non-token data: last expire reduction count Ultimately, it seems that I should be trying to figure out how, exactly, Amavis is calling SpamAssassin in the course of normal operation. Thanks for any help here, folks! -Ben

Calling SpamAssassin parse function from perl program

2010-08-18 Thread Sabiha Fathima
Hi All, Am calling the parse function of SpamAssassin module. Am passing the html message along with all the message headers. The program returns the same input for all the messages, though the message data and the message headers contain spammy content. Please let me know if there are any change

Re: Calling SpamAssassin from a Perl Web Form

2010-08-12 Thread Mike Tonks
> I've yet to hear anyone implementing SA for forms in a sensible manner.. Thanks for the feedback. If people have tried before it's unlikely I'll do much better :) > It would make much more sense to me to just apply well known form spam > specific checks into your code. The standard captcha, to

Re: Calling SpamAssassin from a Perl Web Form

2010-08-05 Thread Henrik K
On Thu, Aug 05, 2010 at 10:58:05AM +0100, Mike Tonks wrote: > Hi folks, > > I'm looking into hooking the Mail::SpamAssassin module into a perl > processor for a couple of web forms - contact us, comments form, and > publish an article form (open publishing). I've yet to hear anyone implementing S

Re: Calling SpamAssassin from a Perl Web Form

2010-08-05 Thread Bowie Bailey
On 8/5/2010 5:58 AM, Mike Tonks wrote: > Hi folks, > > I'm looking into hooking the Mail::SpamAssassin module into a perl > processor for a couple of web forms - contact us, comments form, and > publish an article form (open publishing). > > The main barrier seems to be the need for a message form

Calling SpamAssassin from a Perl Web Form

2010-08-05 Thread Mike Tonks
Hi folks, I'm looking into hooking the Mail::SpamAssassin module into a perl processor for a couple of web forms - contact us, comments form, and publish an article form (open publishing). The main barrier seems to be the need for a message format rather than just a plain text body. I tried two

Changing scores/rules on the fly when calling SpamAssassin from MailScanner

2007-07-11 Thread Matt Hampton
Hi I am looking at writing an extension to MailScanner so that we can allow different settings to be applied. My primary objective is to allow different username to be used for bayes. If I am able to achieve scores and and rules as well this would be a bonus. I have mocked something up which us

Re: Calling Spamassassin

2007-06-22 Thread Bazooka Joe
sendmail -> spamass-milter -> spamd - because I needed to be a spam/virus filter relay for other mail servers. Also, I like being able to reject mail if SA deems it spam. That way if it is a false positive then the sender is aware that the end user did not get the mail instead of going into a s

Re: Calling Spamassassin

2007-06-22 Thread jdow
y stupid things - like play a distinctive sound file when I receive email from certain customers as an alert.) {^_^} - Original Message - From: "Thomas Mullins" <[EMAIL PROTECTED]> To: Sent: Friday, 2007, June 22 06:12 Subject: Calling Spamassassin Just curios to see how pe

Re: Calling Spamassassin

2007-06-22 Thread Mark Martinec
> Also, that way outgoing mail is not driven thru SpamAssassin, > which is good. Depends. In my experience it is not good. Running outgoing mail through SpamAssassin has some advantages: - prevents internal infected/zombiized hosts from spewing their stuff; - presents quality examples of ham to b

Re: Calling Spamassassin

2007-06-22 Thread Jari Fredriksson
>Just curios to see how people call SA? And, why did you choose one method over >another? We have been using Amavisd to call SA for the last three or four >years. >Recently, I have tried out the daemonized version of SA. Using spamd, and spamc from /etc/maildroprc That way I can use different

  1   2   >