Yay, a long-ish post. But I believe it's worth it.
On Tue, 2009-05-12 at 13:14 -0700, an anonymous Nabble user wrote:
> Karsten Bräckelmann wrote:
> > The problem is with the design itself. Only the real sender can and will
> > confirm. The challenge to the *forged* sender of spam will not be
> > responded to. Good for you, bad for everyone else. You are sending
> > backscatter! Spammers are using the very same addresses the are spamming
> > as the sender. Thus your glorious solution to finally end the spam once
> > and forever is SENDING SPAM to innocent humans, bystanders, mirroring
> > your spam to them.
>
> No more than the standard backscatter you get when someone has joe-jobbed
> your email address. Every couple of months, for a day or two, I'll get
> 400-500 bounces a day sent from my hijacked address.
See, that's exactly the point. What is that backscatter? Badly setup
SMTPs, first accepting the mail, then checking and bouncing spam. And
any type of C/R verification and other auto-responders.
Every single newly setup C/R system will add to the backscatter load.
The more folks argue "hey, others are backscattering C/Rs, so why
shouldn't I" the worse it gets. And yes, it *adds* to the noise.
> > Seems to explain why you're using Nabble instead of subscribing to the
> > mailing list. You do not want my email.
>
> But it'd be whitelisted, I think. And the only reason I use nabble is that
Whitelisted? Maybe. If you do it manually. Well, you need to COI your
subscription request, so there'd be the first hurdle. But what about
off-list hints I'd prefer to sent privately?
> often technical lists are full of people who spend their time doing weird
[...]
> Anyway, I digress..
Indeed. And IMHO you'd better not judge about or insult the hackers that
are willing to help you. ;) And FWIW, that is absolutely not my
impression. Anyway, back on-topic.
> > I seriously hope you're not getting much help on integrating such a
> > horribly C/R backscatter beast. If, instead, you are willing to drop
> > boxtrapper and need help with SA, we'd be glad to assist.
>
> OK, you make the point very clearly! So, if I don't use the boxtrapper
> method, what do I do with the 1 in 10,000 emails which is a false positive
> and isn't on my whitelist? How do I give that email the extra chance?
Thanks. :) Part of the problem and my statement is, that around here
we're every now and then discussing exactly that problem. Backscatter.
You'll find the guys active in here will never ever do that. We are,
however, victims.
What about the *rare* FPs? Well, no system is perfect. If you want to
reject spam (at the SMTP level, not after accepting!), do it at a score
above the required threshold. Say, a score of 10 or 15. Or just dump
them in some dedicated folder, to be searched if need be only.
Everything between the spam threshold (default 5) and that high value
simply goes to a folder for review (by Subject generally is sufficient).
With a second threshold of 10, you should be getting < 1% of your spam
in that review folder. And it's done quickly, because you only need to
look out for potentially good stuff.
Ah, and probably try not even to filter known good mail, like mailing
lists, through SA.
Btw, in my experience your estimate is quite correct. If you're getting
significantly more FPs in 10k hams (not overall), you've customized SA
much too aggressively. The default SA scores are optimized to avoid FPs
like the plague at the cost of /slightly/ more spam slipping through.
> >> And while I'm here, how come there's a large and rapidly growing binary
> >> file in /home/myaccount/.spamassassin/auto-whitelist which currently
> >> has 5mb of spammy addresses I've never emailed?
> >
> > It's the AWL, a historical score averager [2] for the senders addresses.
>
> But AFAICS (or the docs suggest) this is for whitelisting email addresses
> sent FROM my account, but the logs show my account definitely DIDN'T send
> this email.
No, it isn't. For the moment, ignore the "white" in AWL. It really is
just a historical score averager, keeping track of previous SA scores
per sender and netblock.
Unless you feed your own, sent mail through SA, AWL doesn't know about
anything like you mentioned above. It does know about senders of
everything it scans, which usually is the incoming mail.
Now what's with that "white"? The original idea is, to allow senders who
generally send low scoring mail to occasionally send something spammy
looking without going beyond the spam threshold. This historic averaging
typically doesn't apply to spam, because the sender is forged, mostly
unique, and the originating netblocks are widely spread. Unlike with
real, good mail.
> >> Thanks, and apology for the length of this, but over the 1.5 months I've
> >> been battling this, I've built up quite some info!
> >
> > Unfortunately, that info didn't include some real hints about the
> > challenge response sender-verification pest, and why it is BAD.
> >
> > Please, do NOT use challenge response sender-verification, do NOT use
> > boxtrapper.
>
> I can tell you're hinting at something...just say what you mean :)
I did. Please do not use a C/R verification system. And explained, why
you shouldn't. :)
> Anyway, thanks for the help on this. You've persuaded me - I'll find another
> method than BoxTrapper. It'll still involve spamassassin, I just can't think
> of how to catch the false positives.
Yay. :) Good.
Just use SA. Start with the default. Tweak it a little, you'll find
plenty of repeated advice for good third-party additional stuff in the
last couple weeks archives. And don't whitelist, unless necessary.
Usually, there's absolutely no need for it.
There shouldn't be any real FPs, and you shouldn't worry about it right
now. A low-volume review folder makes it easy to grab the very rare FPs.
You'll find you will not need a backscattering line of defense.
guenther
--
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
