Joe Flowers wrote: > Matt Kettler wrote: > >> The only problem I see with this approach is that it treats false >> positives and >> false negatives as being equally bad. >> >> > > We do get many more false negatives than false positives, even though we > don't get false positives very often - they are rare. > We certainly don't get 1 fp for every fn. > >> In general, you're adjusting the score bias so the number of FP's and >> FNs are >> approximately equal. > > > This is not what we are seeing in practice. It's not even close to 50-50. >
Based on JM's comments about the score distribution for hams being non-linear, this makes sense. If the distribution was linear for both you'd get 50/50 by dividing the score between the two means. Since the ham is going to have a pretty sharp drop-off somewhere slightly above it's mean your split score approach won't be as bad as 1:1, but it's also likely to not be as good as 100:1 which the 5.0 threshold should get you. It's an interesting concept, and it would be very interesting to graph out FP vs FN rates against thresholds. This graph from JM's post is real data: http://spamassassin.apache.org/presentations/HEANet_2002/img12.html But it doesn't go below 5.0. It would be interesting to see how those curves continue as you approach 0. This graph is a good conceptual one in the "normal" sense of numbers: http://taint.org/xfer/2005/score-dist-doodle.gif That graph would suggest that somewhere below 5.0 there is a threshold at which the ham FP rate gets MUCH worse in a very sudden way. However, there's no score associated. I'd venture to guess that your "average of the means" is going to wind up picking something near, but just above that threshold. That's a bit of an intuitive guess, but also it has some roots in reality. The average score of a ham message on a curve like that is going to wind up being somewhere in the middle of that nasty drop off. By biasing just above that you should bring yourself into the second part of the curve, where decreases in score have a somewhat modest impact on FP rate.
