Re: [Declude.JunkMail] Negative weighting filters to reduce false positives

[Darin Cox]

Well you make a good point on the value of NOABUSE and NOPOSTMASTER.  NOABUSE hits on about 18% of incoming mail, while NOPOSTMASTER hits on about 10%.  Most of the false positives we see from them can be eliminated with these filters, extending their usefulness....so there's still value in them for us.   Certainly REVDNS would avoid forging issues... we just went with MAILFROM initially since many of the false positives we saw were coming from various mailservers, some in the domain, some not....but the filter files could mix MAILFROM and REVDNS as desired, depending on the patterns seen. Darin.     ----- Original Message ----- From: Scott Fisher To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 6:03 PM Subject: Re: [Declude.JunkMail] Negative weighting filters to reduce false positives I myself have pondered why I am even running the RFCI-noabuse and RFCI-nopostmaster test. The NoAbuse misfired 23.6% of the time. The NoPostmaster misfired 12.7% of the time. Due to the underperformance, I weight each test 5 (hold at 200).   The test failures are who's who of ISP / webmail providers. The vast majority of the results are from e-mails with faked mailfrom addresses... Zombie spammers.   I wonder if you'd be better of using the REVDNS instead of the Mailfrom. ----- Original Message ----- From: Darin Cox To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 4:41 PM Subject: [Declude.JunkMail] Negative weighting filters to reduce false positives We just started something I've been thinking about for a while:  Negative weight tests to offset specific test failures for well-known domains.  For example, a large number of false positives we see are from Earthlink, Mindspring, Sprint, Verizon, etc.   Now you may be thinking, of course, these are large providers with dial-up user bases, so you would expect a large percentage of false positives to be from them...but hold on a minute.  Many of these large domains are being penalized in our system for routing or not having abuse@ or postmaster@ addresses.  Almost all of these would not have ended up in the hold queue if they had not been so penalized...thus the idea to figure out a manageable way to NOT penalize them for these technical RFC violations.   So, what we've done is to start filters to counteract the weights for major tests that a few of these domains fail.  By doing it specifically for a particular domain, we reduce false positives but avoid losing the effectiveness of the test on other domains.   Anyway, attached zip are the filter files.  As I mentioned, they have just been started, so there are just a few domains in them at present.  At the top of the filter file are suggested guidelines on how to use them.  There are probably better ways to handle this, so I welcome comments/feedback. Darin.