On 6/8/2018 12:02 PM, David Hofstee wrote:
Earlier you stated that larger setups depended on having blacklists at
the gate to keep processing manageable but which results in less
weighed filtering. [see #5]
yes, but I wasn't referring to ALL blacklists - just a handful of the
most effective and reliable ones that have extremely low FPs - and then
the others would ideally be using for scoring
But these setups (often) still lack feedback mechanisms that the
Yahoo's, Hotmail's and Gmail's of this world have... Feedback from
users. It also lacks methods that need large quantities of "historical
data". Both add information back into the content filter to improve it.
fwif, while, not perfect, invaluement has a number of such feedback
mechanisms in place that constantly help improve the quality and
effectiveness of our data. (I really don't want to provide a more
specific/detailed answer.)
Hey Rob, I think that you have a very good blacklist. My point is that
I think "an excellent blacklist is not good enough". Because you need
a better (and faster) integration between the content filter (using
e.g. reputation data from you), the traffic limiting mechanisms, the
feedback mechanisms (e.g. users marking as spam, or the other way
around) and "email history of your domain".
It is precisely BECAUSE invaluement DOES have things... that helps
invaluement to be "a very good blacklist" (though there is always room
for improvement, and what you envision might be even more comprehensive
than what we do/have!). But some of your suggestions may require
paradigm changes in how spam filters and blacklists interact. There are
probably some good ideas there, but coordinating these with distributed
hardware and software spam filtering is no trivial task. (it is bad
enough that so many spam filters STILL don't have a way to add custom
3rd party URI blacklists - they just hardwire in SURBL, and you can't
add URIBL or Spamhaus' DBL or ivmURI - and this technology has been
around for 13 or so years, so things like this are not easy!)
> (6) nobody in this thread ever claimed that blacklists-alone are
sufficient for having good spam filtering
Rob, you as a small set of people, are capable to and have enough
access to improve on the current situation. I also think that the
Google's, Microsofts and Yahoo's are a systemic issue in the email
world. This would include Proofpoint and SpamExperts as well. It is
becoming harder and harder to have your own server and not be troubled
by spam unless you use this small set of services. "They" are
unwilling to share their methods with the rest of us (which I can
understand but which results into "the rest" not having that knowledge).
I think that we, "the rest", should develop better ways to filter
spam. That was my goal of this discussion. To make you conscious of
the fact that we need more than IP/domain blacklists. To be able to
level up with the proprietary solutions. Instead of being stuck with
the idea that we should stick to what we have (because we have it).
Fair points. but, (1) invaluement is priced reasonable fwiw, so
hopefully that helps! -AND- (2) if you only knew the blood-sweat-tears,
over many many years, required to accumulate this knowledge and
technology! Plus, there are many risks involved too! (even occasional
death threats, and a constant litigation liability) Again, fwiw... not
to complain, and I think that invaluement is going to pay off amazingly
well in the not too-distant future, and eventually make this whole
effort extremely financially rewarding - but a few important pieces to
the puzzle need to come together first. I'm working on that. In the
meantime, I don't think anyone should be very envious of my situation
(morbid fascination would probably more appropriate!) But these may be
the reasons why there are not as many high quality DNSBLs.
--
Rob McEwen
https://www.invaluement.com
_______________________________________________
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
