From: Alex <mysqlstud...@gmail.com> >On Mon, May 1, 2017 at 8:44 AM, David Jones <djo...@ena.com> wrote: >> From: Alex <mysqlstud...@gmail.com> >> >>>I also have a few questions about other rules that hit this email as >>>well as some other rules I've come across today that I don't >>>understand. Most of the questions relate to scoring appearing to be >>>very high for the single rule. >> >>> * 1.4 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/) >> >>>This rule hits messages with an empty body. We receive a lot of mail >>>with invoices, PDF and other attachments with an empty body. Doesn't >>>1.4 points seem a little high just because there is nothing in the >>>body? >> >> I have this same problem and solve it with custom meta rules that >> shortcircuit as ham. Reputation-based rules mentioned yesterday >> also help with this to subtract points for trusted senders.
>You seem a lot less reluctant to whitelist or shortcircuit than I am - >I'm more concerned about allowing PDF spam, then never knowing about >it until it's reported by a user. If your SA instance doesn't filter for any user/human mailboxes that can get compromised, then you can whitelist or shortcircuit all outbound mail. My filters send millions of emails outbound each week and I have customer mail servers out of my control that get compromised accounts often. If I didn't have tight outbound filtering then my servers would be consistently listed on RBLs. Here's one of my 8 mail servers listed on no RBLs and a senderscore.org score of 98 out of 100. http://multirbl.valli.org/lookup/96.4.1.10.html >I've taken a more conservative, but also more time-consuming approach >by creating rules that subtract a few points with the right >combination. That's exactly what I was recommending by creating meta rules with ALL_TRUSTED. I have a lot of customer scanners/copiers that send email that look very spammy with missing/invalid headers so I made a shortcircuit rule with some regex to match on common patterns that I saw from many different scanner/copiers that probably all share the same crappy SMTP source code. Now I don't have to worry about blocking customers' scanners/copiers which used to take up a lot of my time whitelisting individually as they were reported. >I was also hoping there was a more general approach that would make >these rules with such high scores less prone to FPs in the first >place, or at least create a greater burden by default before adding >such high scores to rules involving just a regex. I am trying to show how to combine reputation-based rules with existing/ default SA rules that will solve this problem without requiring a lot of baby sitting of mail logs and constantly adjusting scores and rules which is always reactive. I got tired of chasing the latest spam campaigns and compromised accounts so I did some deep analysis of my mail scoring and found that a good sender reputation constantly stood out in ham. It took some time but I setup a good list of whitelist_from_rcvd for those senders that didn't have SPF or DKIM and whitelist_auth more recently since SPF is pervasive these days. After doing that, I can sit back and not constantly react to new spam campaigns. I keep my Bayes trained up every few days with easy drag-n-drop into folders and let it ride. >> * 3.3 MSGID_NOFQDN1 Message-ID with no domain name >This one catches even automated reports generated by HP to many of our >users, as well as a common email fax service. They just don't consider >proper RFC compliance in their shell scripts, and to basically turn it >into spam just for that is unreasonable. Again, if you put the HP reports into whitelist_auth or whitelist_from_rcvd then the problem is solved. >Also unfortunately, they don't comply with SPF or DKIM conventions, >and one might argue simply passing SPF_PASS isn't sufficient for a >meta rule before whitelisting. Depending on the sending domain, SPF_PASS is sufficient for whitelisting. Take a look at facebookmail.com in your logs and see how it scores. No need to keep wasting CPU cycles on those and other emails that regularly score low. If you do some log analysis and see that a sender with SPF_PASS is regularly scoring well below zero, then it's safe to whitelist this domain if it's not from user/human mailboxes that could be compromised. If you look long enough at your logs, you will see a pattern of user/human domains and then a pattern for generated emails that will help you build entries in whitelist_auth and whitelist_from_rcvd. You may contact me off list for the details of my findings but I don't want to publish my findings on this public mailing list. It's not rocket science or anything ground breaking but it's working very well to make my filtering very accurate. 99% of spammers won't send spam this way and have a good reputation (senderscore.org and other RBLs). There are a few that do and I block them via Postfix. BTW, the Invaluement RBL is a huge help for reputation and well worth it's low cost if you have a large mail filtering platform. I have it in my Postfix postscreen RBLs with a high weight and in my SA rules and it is spot on. Never had a false positive in the years I have subscribed to it. Dave
