Le 05/01/2011 17:00, Rob McEwen a écrit : > On 1/3/2011 6:58 PM, mouss wrote: >> as you can see, all DNSBLs but spamhaus are more or less useless. > > Mouss, > > [ignoring content filtering for a moment... per the original poster's > request] > > If one DNSBL removed 90% of all spams, and that made a users's spam go > from 100-per-day to 10-per-day, that is great... but the end users is > STILL stuck with 10 per day. They go on vacation for a couple of days > and they have dozens of spams to wade through. (but that is better than > hundreds!). Next, your competitor used that same DNSBL, but added > another very high quality and low-FP DNSBL that whittled that 10-per-day > down to 2-per-day. Your customer complains about the spam and starts > thinking about switching his service to your competitor after "comparing > notes" with friends who used your competitor's service. Does the > customer even care that much when you explain that you are doing a great > job because you are already blocking 90% of the spam? > > BOTTOM LINE: In this example, this additional 2nd high quality DNSBL was > probably only hitting on a tiny, tiny percent of the total incoming > spam. But that is not always the best measure. We get fixated on the > percentage of spam blocked using all incoming spams as the denominator. > But sometimes it is a superior measure to use "remaining spam in the > user's inbox" as the denominator because that is more of a "real > world.... what the customer actually sees" measure. > > Otherwise, for example, if easy-to-catch botnet spam doubled and was > easily blocked... and, at the same time, hard-to-catch snowshoe spam > also doubled... but was often missed. Then, numbers-wise, using the > incoming spam as the "denominator" in our measurements, we'd all be > patting ourselves on the backs for all the spam we were blocking.. at > the SAME time that the spam making it to the inbox INCREASED > substantially!!! Something would then VERY wrong with our measurements > of success! >
Rob, I fully agree with you. measuring filter "performance" is not easy (I once started trying a "cost" based model...). I sure understand that a list that blocks that little spam which isn't blocked otherwise is a very very good thing (and you can see that I still use spamcops, korea, ... even though they don't catch a lot). but I only tested combinations of freely available lists, and in my post, I omitted a local BL (but it doesn't catch a lot of junk here any more. looks like snowshoers avoid me now?)
