Dennis Hardy wrote:
>Do people generally have good non-FP experience with BRBL? I am
>thinking of bumping up the score, but I get so much spam per day
>it is hard to check for FPs with it enabled.
Dennis, it depends on what sort of ham your people receive.
For evaluation purposes, I've been running Barracuda on three diverse
domains (one Geek, one "pure business", one mixed business&family).
Each maintains a decent-to-excellent hand-classified corpus (they share
summary data with me).
The Geek domain had 2 Barracuda FPs (between 28-Sep-2008 and today).
Both were from the same IP, so I've merely skip listed it.
Unfortunately, that particular sender was a webhost, with one of the
FPs being critically time-sensitive, so I consider those FPs to be
completely unacceptable (albeit easily avoided). Both of those emails
received an SA score of "0.0", so the mentioned score of "3.0" would
NOT have stopped them (that particular webhost is extremely Geeky, and
doesn't commit any HTML atrocities).
The "pure business" domain had a zero Barracuda FP rate (note it's only
been running since 24-Jan-2009).
The other domain (running Barracuda since 16-Jan-2009) receives a LOT
of requested mailing list traffic (Constant Contact, Cheetah, etc), and
has had a significant number of FPs.
Here are the number of Barracuda hits for the last two weeks, for the
domain with FPs:
spam 5005
ham 38
Of the ham IPs, 22 had been previously classified as (generally) legit
bulk mailers (i.e. "ESP"s). Visual inspection of the rest showed that
_ALL_ were some sort of mailing list, mostly business oriented, the rest
charitable or social. When I sorted that data by SA score, it was
uniformly distributed across the different types of senders.
Here is the breakdown of SA scores for the ham:
SA range Hits Percent
<sub zero> 2 5.3%
0.0 - 0.6 7 18.4%
1.0 - 1.8 13 34.2%
2.0 - 2.4 3 7.9%
3.0 - 3.5 7 18.4%
4.0 - 4.2 6 15.8%
During that same period, there were 16 hams that had SA scores above
the cutoff threshold. If I had scored Barracuda at "3.0", the
potential FPs would have doubled.
Note that I am not currently running Barracuda via SA
(I'm doing the testing in a different filter which runs right after SA).
Bottom-line:
Depending on the nature of your ham, you are likely to get some FPs,
even at the mentioned score of "3.0".
If you have a weak FP pipeline, then be very cautious.
Consider scoring Barracuda weakly, and using it in a "meta" context.
If anyone wants it, I can dump the specific SA tests for those FPs, as
well as a separate list of the spam hits (should be useful for creating
meta rules).
I will also update those stats in a few more weeks/months.
