Thank you for this interesting list. Unfortunately my external MTA is based on exim. I think that a lot of the checks you list are already performed by exim and by the rules we added to that MTA. I will check one by one anyway.
Perhaps the most interesting idea, something we were already thinking about, is to move spamassassin away from zimbra and give it a standalone server. We will lose something in integration but we can be free from zimbra release cycles. On Thu, Jun 27, 2019 at 2:38 PM David Jones <djo...@ena.com> wrote: > On 6/26/19 3:43 AM, hg user wrote: > > Thank you everybody for your really interesting answers. In this moment > > I'm just collecting informations. > > > > I have one main problem: one of the engines used by our commercial > > antispam solution returns too many FPs. I'm gradually introducing > > spamassassin (included in zimbra) and I'd like to mitigate the FPs with > > some other checks... using a proven, well-known technology like AskDNS > > seems a quick and viable solution to me. > > > > Unfortunately a personal RBL may not cover all the use cases I'm > > thinking about and looking at the source code of a plugin that queries a > > sql or redis server can be interesting. > > Before you start working on a custom plugin, have you tuned out your MTA > and SpamAssasin? From my personal experience, I setup an edge MTA as > the MX and sent filtered mail to Zimbra and smarthosted from Zimbra back > to the edge MTA. This provides the most flexibility to upgrade perl and > SpamAssassin to the latest version along with many other benefits. > > Tuning out the MTA: > - Setup Postfix with Postscreen > - Enable weighted RBLs in Postscreen, lots of them. See the SA mailing > list archives for "postscreen_dnsbl_sites". > __This will block 80% or more of spam/junk alone.__ > - Setup postfwd to give extra control to add headers based on SMTP > conversation time so SA can use those headers later. For example, I set > headers based on the number of recipients which is very useful when > email has been BCC'd. > - Setup sqlgrey and slowly phase it in where users won't even know it. > - Setup policyd-spf, OpenDMARC, and OpenDKIM > - Setup fail2ban for repeat spammers/bots > - Setup Postwhite to whitelist trusted senders by their SPF record. > This allows for turning up other Postfix config settings > - Setup TLS with a Letsencrypt certificate > - Setup rate limiting then put exceptions in > smtpd_client_event_limit_exceptions. > - Postfix header_checks, body_checks, smtpd_client_restrictions, > smtpd_helo_restrictions, smtpd_sender_restrictions, > smtpd_relay_restrictions, smtpd_recipient_restrictions, > smtpd_data_restrictions in the main.cf can be tuned over time. > - Enable reject_unverified_recipient in smtpd_recipient_restrictions so > Postfix will "look ahead" to Zimbra and not accept invalid recipients. > - > > Tuning out SpamAssassin: > - Make sure your internal_networks and trusted_networks are correct so > RBL checks will happen correctly for the last external IP. I have > extended this out to Google, Office 365, and other major platforms to > detect the X-Originating-IP of the web/mail client. > - Install KAM.cf and KAMonly.cf > - Install DCC, Razor, Pyzor > - Install ClamAV unofficial (extra) signatures > - Add local rules to use the headers from OpenDMARC > - Enable extra RBLs that aren't in the stock SA > - I use the ShortCircuit plugin heavily, disable the ALL_TRUSTED > shortcircuit, and enable shortcircuit on a number of the USER_IN_* rules. > - I have created a massive list of whitelist_auth entries that are > mostly subdomain senders from trusted senders. > - Setup a way to train your Bayes easily by dragging email into a Spam > and Ham folder as things are misclassified to keep the Bayesian DB tuned > correctly. > - Get on the latest version of perl even if you have to compile it > because your OS might be older. > - Install the latest stable version of SpamAssassin. > - Many more things covered on this list over the years. > - I setup local DBLs and DWLs for brand new Office 365 senders and other > common sources of spam like secureserver.net, unifiedlayer.com, > websitewelcome.com, myregisteredsite.com, etc to add a couple of points > for new senders. Then I add good senders on those bad hosting platforms > to a DWL that subtracts a couple of points and excludes them from other > meta rules that amplifies certain scores for the spam. > > Note that a lot of this can be found by setting up a quick VM and > installing iRedMail to check out the Postfix configuration for the > milters mentioned above and the TLS configuration. It uses Amavisnew so > that might be different from how you want to "glue" SpamAssassin into > the MTA. > > I use MailScanner which has a few extra features of it's own in addition > to processing emails in batches for high volume mail flow. > > After I did all of that work above over many years, my mail filtering > accuracy is very good for about 80,000 mailboxes. The more mailboxes > and domains you filter, the more time it takes to tune everything properly. > > > > > > Thank you > > Francesco > > > > On Tue, Jun 25, 2019 at 10:20 PM Matus UHLAR - fantomas > > <uh...@fantomas.sk <mailto:uh...@fantomas.sk>> wrote: > > > > >On Tue, 2019-06-25 at 11:09 -0500, David B Funk wrote: > > >> that's way overthinking it. > > > > On 25.06.19 17:55, Martin Gregorie wrote: > > >I agree, now that there's a configurable OSS dnsbl server > available, > > >that using it is the obvious choice for dealing with a standalone > > list, > > >but the OP did ask specifically about using database queries to > > >implement a blacklist, so I thought it was worthwhile to tell him > > what's > > >involved in doing that. > > > > No. The OP wanted to store data in DB to avoid restarting SA, not > > mentioning > > any other specific reason to use DB. > > > > using DNSBL does avoid restarting SA and does not require any > > plugin, which > > is a great advantage. > > > > we are trying to provide described requirements, while avoiding > proposed > > complicated solutions. > > > > >For all I know the OP either has a similar archive or is intending > to > > >implement one: searching for a specific message with a database > > tool is > > >a *lot* faster than ferreting through a set of very large mail > folders > > >with your MUA, though of course the effort of creating and > maintaining > > >the database, mail loader, query tools and SA plugin is non > trivial. > > > > well, if THIS is the real reason... > > > > -- > > Matus UHLAR - fantomas, uh...@fantomas.sk <mailto:uh...@fantomas.sk> > > ; http://www.fantomas.sk/ > > Warning: I wish NOT to receive e-mail advertising to this address. > > Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. > > Saving Private Ryan... > > Private Ryan exists. Overwrite? (Y/N) > > > > > -- > David Jones >
