Dan, >Then help rally the SA team to include those RBLs >that you mentioned in the stock config.
My RBL (ivmSIP.com) wouldn't work as a default value in SA because it is only available via RSYNC or Zone Transfer to subscribers (or... currently... "testers" who have specifically requested access).
The other weird thing is that I use SA as a "helper app" in my spam filtering and I've custom written my own spam filter. Mostly, I still include SA in the mix for SARE rules (& other rules), as well as checksum filtering like RAZAR, etc. But I've turned off all RBL & URIBL filtering in SA because I do those on my own and, most of the time, SA isn't even needed.
As a result, I pay very little attention to many of the implementation details of "RBLs" in SA since I don't personally use them in SA. I have enough to worry about without these extra details. However, I'll be happy to share some tips that might help others or the SA folks with possible improvements in future versions.
First, one thing that I did years ago (and continue to do) is that I'm always carefully reviewing lists that I might potentially use and/or am already using. For example, if I notice that a particular dnsbl is hitting on more and more messages which ultimately score under the spam threshold and, upon examination, I verify that most or all of these really are legit, them I'm at least going to lower the points assigned to hits on that dnsbl... and I might even remove that dnsbl from my spam filtering altogether.
If, on the other hand, I find that ALL such messages really were spam, I might start increasing the points given to that particular list, assuming that I'm not also seeing some FPs from that list.
Next, if I see a spam (that wasn't sent from a legit ISPs mailserver) and it scored rather low, I'll then take that IP and run it against a spam blacklist checker (dnsstuff, robtex, etc) to see if there are any RBLs that would have caught it, but that I'm not using yet. (Of course, I ignore various FP-ridden lists like APEWS in that search.) If I see a pattern whereby a particular list consistently hits on IPs that scored too low in my spam filtering, I might then add that dnsbl to my filtering... starting off with a low score... then double-checking for FPs... then bumping the score up depending on how little FPs there are. (in this case, I'm calling any "hits" on legit messages a "FP", but, at this stage, these will generate too low a score to outright block and this "FP" really did get delivered to the inbox.)
Doing this, over the years, I've added a good mix of RBLs with very fine tuned scoring (in my own spam filtering program, not referring to SA).
At one point, I noticed that many of the more aggressive dnsbls are really really good at catching new IPs, but have too many FPs. As a result, I have to keep their "score" low. But it seemed such a shame because these IPs were taking too long to get on the FP-safe dnsbls. Then I noticed that, many times, three or four of the more aggressive RBLs would quickly hit on the same spammer's IP, where that IP that wasn't yet on SpamHaus, etc... then... if a few lists hit on that new spammer' IP, chances were, it was worthy of blocking in comparison to if just one list hit on it... so much so that the score really needed to be higher than merely the sum of the FP-risky dnsbl's scores.
As a result, I changed my formula so that I took into account the number of dnsbls that hit on that IP as well as the score. (it was something like.. for every added dnsbld "hit" the overall RBL score would get increased by an additional 10% or 20%)... next, I adjusted down some of the "raw" scores so as to not allow the RBL scoring to get out of control. IOW... the whole really was worth more than the sum of its parts! Get it?
Of course, even then, I have extensive whitelisting of IPs that I have placed in front of this... both my own (that I've put literally thousands of hours into!) and third parties. Currently, my own IP blacklist isn't (yet) on dnsstuff or robtex... but if it something like it were there and produced by someone else... I would have spotted it in that systematic checking that I described and I'd have been thrilled at its results... IOW... I created a product that I myself would have greatly desired to have if it had been created and distributed by someone else. I probably would have been one the first subscribers.. had this been someone else's product. (Why? Because my RBL provides that same "fast reacting aggressiveness"... just without the FPs!)
Still, besides my own RBL's "subscription" barrier to inclusion... other lists which also require RSYNC access would not be able to come "preinstalled" in SA since they too need a little TLC to get up and running in one's spam filtering environment. These couldn't be used "out of the box" without some configuring of various programs on one's server. Something else to ponder.
I hope this is beneficial and helps future SA versions! Doing all of this, I believe I've taken the "RBL" portion of my spam filtering to a level that is beyond what many thought possible.
Rob McEwen
