Automatically blacklisting IP addresses based on invalid recipients is very tricky. A while back, I wrote some scripts to do this; they run on my server now. It's taken a while to "balance" them so they don't accidentally blacklist a server for the wrong reasons. For example, if a remote user mistypes an email address in my domain, I don't want to blacklist their server for it. Also, if someone sends spam through a legitimate server from which I receive non-spam email, I don't want to blacklist it either. As a result, my scripts scan email to invalid addresses, build lists of invalid recipient addresses and sending servers, then crunch the numbers to only block servers that have sent to multiple invalid addresses within a small timeframe. The script then scans all of my users' non-Junk and non-Trash folders to find legitimate servers and removes them from the blacklist. The blacklist is also temporary, so the entries automatically expire after a few days.
I'd have to think it through carefully, but my instinct says adding all of this to spamdyke would be a big job. My script works because all of my domains funnel their "catchall" addresses into a single mailbox, which the scripts can analyze. In this way, I'm basically getting recipient validation (not in real time) and a time-based data store. spamdyke would probably need a database engine to capture the data and run totals/searches quickly. However, you should know that even with my script continually blacklisting hundreds of IPs, the percentage of connections blocked by this system is negligible, usually less than 1% (if any). I have a number of theories why. IP addresses are removed from the blacklist if they are found in messages outside a user's Junk or Trash folders. This is done so that legitimate servers aren't accidentally blocked. However, it also means that when a user receives a spam message but doesn't check their email very often, they can "hold the door open" for more spam. I'm not sure how to fix this problem. Alternatively, if a user deletes a spam message instead of moving it to their "Junk" folder, it is lost and can't be analyzed. Lastly, this system can only stop large mailings from dedicated servers. I think most dedicated spam servers are already on DNS RBLs and the botnets don't send enough spam from individual IPs to trigger this system. I strongly recommend getting this script running on some larger hosts to see what their success rates look like. I have a hunch it might not be worth the effort to add it to spamdyke. Personally, I get much more success from my scripts that automatically create my rDNS blacklist. My theory (so far seems to be correct) is that spammers can easily register thousands of domain names because they can be automatically created from dictionary words and registered in bulk. The spammers usually run web sites on those domains to make them appear more legitimate. However, no one has time to create thousands of web sites, so they run the _same_ site on thousands of domains. My scripts download the websites of sending rDNS domains and compare them with a rather large collection of known spammer sites (mostly stock text and distinctive misspellings). When a match is found, it's automatically blacklisted. As a bonus, the script then checks the entire class C block of IP addresses that contain the matching site. It also continues checking the entire class C for 3 weeks, because spammers love ICANN's "domain tasting" feature and replace their domains within 5 days. This proactive scanning finds an amazing number of sites. My rDNS blacklist currently contains more than 33000 entries. Approximately 10%-50% of all incoming connections are stopped by this filter (on my server). I do get some false positives from this system but so far those have been fairly rare. -- Sam Clippinger Michael Colvin wrote: > That was partly why I was asking... I'm currently NOT filtering based on > valid rcptto, and the arguements for and against both have merits. My > initial concern was against harvest attacks for e-mail addresses. However, > at least John Simpson's validrcptto patch takes that into account and adds > some ability to regulate those types of attacks. The other reason I'm > leaning towards adding that ability is that I think the amount of mail I get > that is to bogus addresses, far outweighs my concern for such harvesting. > The proportion of spammers flooding my servers with random mail, some of > which does get through SpamDyke, at least up to the GreyListing part, is far > greater than I think the number of people out there trying to harvest valid > e-mail accounts, which they'll still have to send spam to via "Clean" > systems to get through the filters! > > Now, one thing I think would be very cool, and it might be something easy to > add is a script that checks greylisted accounts to see if THEY are valid. > If they aren't, it deletes the entry, which would cause it to be greylisted > again. The mail NEVER makes it past SpamDyke. The other benefit to this > would be to build into that same script, the ability to record the source IP > of that invalid e-mail into a local RBL...Basically, building a "Honeypot" > system into SpamDyke. It resolves the "INODE" issue I've seen raised here, > and adds a really useful function to SpamDyke, the ability to build a local > RBL based on ACTUAL spam received by YOUR mail system, not someone > elses...Use a local RBL, make it the first RBL checked, you kill the SMTP > session the next time around when that spammer uses that IP... > > To me, that's a better way to handle email to bad addresses..At least get > SOME use out of them. I say this because, the current "Method" of spamming > that I'm seeing is the spammer using a series of IP's. It may be 10 or 20 > different IP's, but he'll use them 3-5 times each, then move to the next, > eventually coming back to the first and cycling through them again. > Sometimes SpamDyke blocks them because they are already on an RBL, sometimes > not. Usually, at first, they aren't, but it may take hours for them to > start getting listed... It's like they find new IP's, start a SPAM run, and > then stop when they're all blacklisted, and then find more and start over. > > Anyway...Just my .02. It would solve a couple existing minor issues, add > some additional functionality, and possibly avoid other issues (The > harvesting). Besides, there are other methods to do valid e-mail checking, > although some require patching, others do not... > > > > Michael J. Colvin > NorCal Internet Services > www.norcalisp.com > > > > > > > > >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of >> Olivier Mueller >> Sent: Thursday, May 15, 2008 4:27 PM >> To: spamdyke users >> Subject: Re: [spamdyke-users] Next Version >> >> Hi, >> >> On Thu, 2008-05-15 at 15:36 -0500, Sam Clippinger wrote: >> >>> I haven't yet decided how to implement it, but I think it won't be >>> useful unless it supports (at minimum) plain vanilla qmail, >>> >> qmail with >> >>> vpopmail, qmail with vpopmail+MySQL and Plesk >>> >> (qmail+chkuser). (This >> >>> feature could finally be a reason to integrate database >>> >> support.) If >> >>> anyone has any other configurations that should be >>> >> supported (LDAP?), >> >>> now is the time to speak up -- let's discuss it before I >>> >> start writing code. >> >> Just an opinion: maybe this doesn't have to be *in* spamdyke? :) >> >> I guess most of the admins around (but YMMV of course) are >> already doing recipient validation, and so you won't have to >> reinvent the wheel and have time for more interesting >> features... For vmailmgr-based systems, there is already >> mailfront+cvm-vmailmgrd, and it's working quite fine. >> I suppose there is the same kind of thing for vpopmail & co. >> >> regards, >> Olivier >> >> _______________________________________________ >> spamdyke-users mailing list >> [email protected] >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> >> > > _______________________________________________ > spamdyke-users mailing list > [email protected] > http://www.spamdyke.org/mailman/listinfo/spamdyke-users > _______________________________________________ spamdyke-users mailing list [email protected] http://www.spamdyke.org/mailman/listinfo/spamdyke-users
