Re: (newbie question) Increasing SA effectiveness

2008-12-12 Thread John Hardin
On Fri, 12 Dec 2008, Marcin Krol wrote: John Hardin wrote: On Thu, 11 Dec 2008, Karsten Br�ckelmann wrote: > I still recommend initial training, to give Bayes a good kick-start. Initial _manual_ training. Define manual: manual picking out spams is plain too labor-intensive. Manual trai

Re: (newbie question) Increasing SA effectiveness

2008-12-12 Thread Kai Schaetzl
Marcin Krol wrote on Fri, 12 Dec 2008 10:37:31 +0100: > Define manual: manual picking out spams is plain too labor-intensive. If > we redefine "manual" to mean ham coming from authenticated mail, and > spam coming from spamtraps, I wholeheartedly agree. The point is that you need to have a corp

Re: (newbie question) Increasing SA effectiveness

2008-12-12 Thread Marcin Krol
John Hardin wrote: On Thu, 11 Dec 2008, Karsten Br�ckelmann wrote: I still recommend initial training, to give Bayes a good kick-start. Initial _manual_ training. Define manual: manual picking out spams is plain too labor-intensive. If we redefine "manual" to mean ham coming from authentic

Re: (newbie question) Increasing SA effectiveness

2008-12-12 Thread Marcin Krol
Karsten Bräckelmann wrote: Well, isn't it better to use them before SA, provided your MTA does have this feature (I recommend Exim to everyone)? No -- unless you ultimately trust the RBL to produce a *negligible* amount of FPs. Every single RBL does have FPs to a highly variable degree. Instead

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Kai Schaetzl
Matthias Leisi wrote on Thu, 11 Dec 2008 22:05:34 +0100: > (and > are thus likely to be quoted in reply emails) correctly working email programs leave the signature out from quoting Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Matthias Leisi
Mark Martinec schrieb: > or construct custom rules to whitelist (=add negative score points) > based on some other specific chraracteristic of mail to be passed. Your own (your companys) street address, phone number, or some hopefully unique token which you typically add in footers of outgoing e

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 18:36 +0100, Matus UHLAR - fantomas wrote: > > > Ned Slider wrote: > > > > Yes, additional DNSBLs such as psbl and uceprotect can be integrated > > > > into SA > > > On Thu, 2008-12-11 at 15:19 +0100, Marcin Krol wrote: > > > Well, isn't it better to use them before SA, prov

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Henrik K
On Thu, Dec 11, 2008 at 05:57:10PM +, Ned Slider wrote: > > Genuine spam traps are great for bayes training as they should contain a > representative sample of spam your users will be seeing plus you know > they only contain spam so you don't need to check the contents before > feeding th

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread mouss
Ned Slider a écrit : > Genuine spam traps are great for bayes training as they should contain a > representative sample of spam your users will be seeing plus you know > they only contain spam so you don't need to check the contents before > feeding them to bayes to learn :) > you must be careful

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Ned Slider
Marcin Krol wrote: Matus UHLAR - fantomas wrote: - blocking at MTA by RBL or other techniques (such as graylisting) is efficient and effective, but deprives SpamAssassin of spam samples, so if your resources permit, it is better to let SpamAssassin deal with all RBLs. I don't think so. W

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Ned Slider
Karsten Bräckelmann wrote: On Thu, 2008-12-11 at 15:19 +0100, Marcin Krol wrote: Ned Slider wrote: Yes, additional DNSBLs such as psbl and uceprotect can be integrated into SA Well, isn't it better to use them before SA, provided your MTA does have this feature (I recommend Exim to everyone)?

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Matus UHLAR - fantomas
> > Ned Slider wrote: > > > Yes, additional DNSBLs such as psbl and uceprotect can be integrated > > > into SA > On Thu, 2008-12-11 at 15:19 +0100, Marcin Krol wrote: > > Well, isn't it better to use them before SA, provided your MTA does have > > this feature (I recommend Exim to everyone)? On

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 15:19 +0100, Marcin Krol wrote: > Ned Slider wrote: > > > Yes, additional DNSBLs such as psbl and uceprotect can be integrated > > into SA > > Well, isn't it better to use them before SA, provided your MTA does have > this feature (I recommend Exim to everyone)? No -- unle

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 08:28 -0800, John Hardin wrote: > On Thu, 11 Dec 2008, Karsten Bräckelmann wrote: > >>> I still recommend initial training, to give Bayes a good kick-start. > >> > >> Initial _manual_ training. > > > > Err... Yes! :) > > The reason I stressed that is it sounds like the OP tu

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread John Hardin
On Thu, 11 Dec 2008, Karsten Br�ckelmann wrote: On Thu, 2008-12-11 at 08:18 -0800, John Hardin wrote: On Thu, 11 Dec 2008, Karsten Bräckelmann wrote: I still recommend initial training, to give Bayes a good kick-start. Initial _manual_ training. Err... Yes! :) The reason I stressed that

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 08:18 -0800, John Hardin wrote: > On Thu, 11 Dec 2008, Karsten Bräckelmann wrote: > > > I still recommend initial training, to give Bayes a good kick-start. > > Initial _manual_ training. Err... Yes! :) -- char *t="[EMAIL PROTECTED]"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 16:28 +0100, Marcin Krol wrote: > Karsten Bräckelmann wrote: > > Do train false negatives. It does help Bayes, if you train "FN according > > to Bayes", that is spam that has been caught, but got a low, ham-ish > > Bayes score. > > It seems that I need to brush up on specific

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread John Hardin
On Thu, 11 Dec 2008, Karsten Br�ckelmann wrote: I still recommend initial training, to give Bayes a good kick-start. Initial _manual_ training. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key:

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Marcin Krol
Karsten Bräckelmann wrote: Do train false negatives. It does help Bayes, if you train "FN according to Bayes", that is spam that has been caught, but got a low, ham-ish Bayes score. It seems that I need to brush up on specifics of SA Bayes; so far I have used only DSPAM from among statistical

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Marcin Krol
Matus UHLAR - fantomas wrote: - blocking at MTA by RBL or other techniques (such as graylisting) is efficient and effective, but deprives SpamAssassin of spam samples, so if your resources permit, it is better to let SpamAssassin deal with all RBLs. I don't think so. We get "enough" of sp

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Matus UHLAR - fantomas
On 11.12.08 15:47, Mark Martinec wrote: > Quality of bayes auto-learning improves if you let all your mail > pass through SpamAssassin: > > - outbound mail is often a high-quality source of ham > for autolearning; But when one of your users starts spamming (trojan or wtf), you have problem and

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 16:01 +0100, Karsten Bräckelmann wrote: > On Thu, 2008-12-11 at 15:13 +0100, Marcin Krol wrote: Forgot to add... > > No, I just waited until default 200 hams and 200 spams kicked it in. As > > I mentioned elsewhere, I get a weird effect of correct positives, but > > relati

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 15:13 +0100, Marcin Krol wrote: > Karsten Bräckelmann wrote: > > Razor is quite good, too. Also Pyzor, though it requires much more > > resources. > > See, my friend who works at a hosting company didn't find Razor to be > much improvement. Perhaps he misconfigured it or s

RE: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Bowie Bailey
Marcin Krol wrote: > Karsten Bräckelmann wrote: > > > > Did you manually (initially) train it > > with your collected ham and recent (not older than 3 months) spam? > > No, I just waited until default 200 hams and 200 spams kicked it in. > As I mentioned elsewhere, I get a weird effect of correct

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Mark Martinec
Marcin, > >Did you manually (initially) train it > > with your collected ham and recent (not older than 3 months) spam? > > No, I just waited until default 200 hams and 200 spams kicked it in. As > I mentioned elsewhere, I get a weird effect of correct positives, but > relatively many false negati

RE: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Bowie Bailey
Marcin Krol wrote: > Matthias Leisi wrote: > > > * If circumstances permit, make use of extensive whitelisting, so > > that you can increase the score of rules (or maybe lower the > > threshold after which you consider a message to be spam). > > With all due respect, that's risky... My users ofte

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Matus UHLAR - fantomas
> Ned Slider wrote: > >Also look at setting up Bayes and train it well. A well trained Bayes > >setup can hit 99% plus spam (for me) and can be highly effective. On 11.12.08 15:19, Marcin Krol wrote: > Except I found that while it often gets positive identification right, > it sometimes produces

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Marcin Krol
Ned Slider wrote: Yes, additional DNSBLs such as psbl and uceprotect can be integrated into SA Well, isn't it better to use them before SA, provided your MTA does have this feature (I recommend Exim to everyone)? Also look at setting up Bayes and train it well. A well trained Bayes setup can

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Marcin Krol
Karsten Bräckelmann wrote: - SURBL and URIBL are extremely effective at identifying spam They are enabled by default -- unless you are running local tests only. Did you (or your distro default) disable network tests? If you specifically had to enable these, you are likely missing more of them.

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Marcin Krol
Matthias Leisi wrote: * If circumstances permit, make use of extensive whitelisting, so that you can increase the score of rules (or maybe lower the threshold after which you consider a message to be spam). With all due respect, that's risky... My users often get legit mails out of blue or e

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Mark Martinec
> * If circumstances permit, make use of extensive whitelisting, so that > you can increase the score of rules (or maybe lower the threshold after > which you consider a message to be spam). When whitelisting, never whitelist just based on a plain sender or author address (such as 'whitelist_from'

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Karsten Bräckelmann
On Thu, 2008-12-11 at 12:52 +0100, Marcin Krol wrote: > Through experimentation I have found that the following techniques are > highly effective: > - SURBL and URIBL are extremely effective at identifying spam They are enabled by default -- unless you are running local tests only. Did you (or y

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Ned Slider
Matthias Leisi wrote: Marcin Krol schrieb: Is anybody here willing to share other / better techniques and tips? No silver bullet, only blood, sweat and tears :-) I agree. * Create custom rules that to match your uncaught spam (and maybe share these rules back on this list). Yes, cust

Re: (newbie question) Increasing SA effectiveness

2008-12-11 Thread Matthias Leisi
Marcin Krol schrieb: Is anybody here willing to share other / better techniques and tips? No silver bullet, only blood, sweat and tears :-) * Create custom rules that to match your uncaught spam (and maybe share these rules back on this list). * If circumstances permit, make use of extensi

(newbie question) Increasing SA effectiveness

2008-12-11 Thread Marcin Krol
Hello everyone, I'm (somewhat) new to SA, and it works nicely, except now I would like to boost its effectiveness at finding spam. I have searched the web and frankly I'm disappointed with the results - except basic config there is not much info there on how to finetune SA to get better result