Okay, since everyone really desperately wants to know, I ran the numbers
on the effectiveness of RBL, RSS, DUL and ORBS against the mail intake for
lists.debian.org. All of this is theoretical and done offline against the
log file, we are blocking only via RBL (and now RSS)
The period of analysis was 1 week.
Stat #1
Of 3054 unique IPs 386 are in one of the RBL's, the breakdown is:
RBL - 16
RSS - 45
DUL - 49 [17 rcn.com, 14, psi.net]
ORBS - 314
Comparing connections it is found that 3970 out of 40236 connection
attempts would have been blocked. This can be roughly considered to be
3970 emails blocked.
Stat #2
Cross referencing the IP list against the bad bounce log shows 13 IPs.
These are highly likely to be legitimate emails.
Stat #3
Cross referencing the IP list against the content filtered spam log
shows 0 hits [not surprising, this log is very small].
Stat #4
Taking the list of all subscriber domains and substring matching this
against the list (loosly, check for people who are blocked but
subscribed to the list) gives 226 matches. Breakdown:
RBL - 1
RSS - 12
DUL - 26
ORBS - 196
The RBL and RSS hits show a very good chance of actually being
legitimate list subscribers :< It is impossible to tell with DUL if
the host is a subscriber on a modem or something else. ORBS is to
prolific to check by hand.
Stat #5
Collecting IPs from all recived and relayed (ie good) list mail and
corellating gives 28 matches. Breakdown:
RBL - 0 [Expected, we are banning RBL]
RSS - 1
DUL - 18 [17 from a single user on rcn.com]
ORBS - 10
Note, during the 1 week period I estimate that no more than 5 unique
spams were recieved. May of the spams were sent to all lists. Also
note that aliases like [EMAIL PROTECTED] are not covered by these
stats.
There seems to be a huge mismatch between messages accounted for and
messages taken in, I think these are due to sucessfully processed bounces
by the list software, which do not get logged [?]
Conclusions
I have been unable to conclusively show that any of the RBLs are actually
reducing spam, but I have positively confirmed that they *all* (save RBL
which I cannot check since we block on it) would result in legitimate
messages being blocked.
ORBS deserves special mention because of their insane hit count, I don't
know what that is about but ORBS would block 10% of the mails we get. I
think it is without question that the majority of those blocks are
legitimate mails. ORBS is also almost completely inclusive of the RSS and
RBL.
DUL would seem to effect at most maybe 10 people, but it hasn't actually
been shown to stop any spam - so this needs more investigation. DUL has a
policy that many people find objectional.
A perusal of the DUL ips all suggest they are *all* modems which is a
really selective filter swath. No DSL or Cable IPs appear to be listed!
RBL has not been conclusively shown to stop spam, but it has such a low
impact (<3 uniq hits each day) that we use it anyhow.
RSS has been observed to list the occasional spam, this is expected since
they respond to spammer activity - but it is also shown that it will
effect at least 1-2 people.
* Note, once a site is listed in one of these RBLs it becomes impossible
for a user to unsubscribe from our lists - no matter what they do they
will never be able to communicate a bounce or a unsubscribe request - this
is pretty bad.
Jason
