This is TERRIFIC. Terrific, terrific, terrific.
I've done a lot of regex work in my days mostly in php, vb, and linux batch
scripts, but wasn't aware of the *FAIL syntax. I was thinking a return of
1 or 0, not no return. That makes much more sense.
And thanks for the continued examples. I'd be shocked if more than 10
people out of the couple thousand remaining ASSP users (??) has an
CorrectASSPcfg functions in a regex.
Three small charities that used to use my charity's ASSP installation for
email filtering have gone direct to Office365 and removed ASSP from
the equation in the last 2 years. I'm still in touch with their staff.
While the like 365's features, they can't stand the spam filtering. You've
spoiled them with never having to sort through a junk folder!!
On Fri, Jul 15, 2022 at 6:37 AM Thomas Eckardt <thomas.ecka...@thockar.com>
wrote:
> Ken - learn perl regular expressions!
>
> >Why is the *FAIL bit in your example of
>
> - the (*FAIL) or (*F) statement makes a regex fail, even a match was found.
>
> Because the return value (e.g. setting $_) from a code execution in a perl
> regex does not modify the 'match found/no match found' flag of the regex.
> But the $_ can be used in a conditional regex to tell the regex engine what
> to do (in which case).
> There are multiple ways to do it. And possibly there are better ways - but
> this one I found nice.
>
> (\@.+\.docusign\.net|next domain|next
> domain|...|...)(?(?{&CorrectASSPcfg::myWantedDKIMCheck($fh,$+)})|(*FAIL))
>
> explanation:
> ( the matching strings/domains, match captured in $+) # if failed, the
> next parts of the regex are ignored and the regex fails - if matched 'match
> found' is set by the regex engine
> (?( # start of a
> conditional (yes|no) regex (?(cond)yes|no)
> ?{ # start of the
> code to be executed ?{code}
>
> &CorrectASSPcfg::myWantedDKIMCheck # call this sub
>
> ($fh,$+) # provide the filehandle and the last match
> result to the sub
>
> }) # end of the code and the condition - the
> return value of the sub is the conditional result
>
> | # the 'yes' part (before the pipe [empty]) -
> if the code returned 1, nothing is to do, regex keeps 'match found'
>
> (*FAIL) # the 'no' part (after the pipe) - if
> the sub returned 0 or undef make the regex fail (no match found)
>
> ) # end of the conditional regex
>
>
> > seems to return if there's no DKIM (return unless $this->{isDKIM};)
> wouldn't that not match the regex, so the 60 score wouldn't be applied?
>
> right!
> returns undef in case there is no DKIM-signature found - which makes the
> regex fail (*FAIL) -> no score
>
> If you need it the opposit way change the return value of the sub - or
> the regex ....})(*FAIL)|) <- here the 'yes' part sets 'no match found'
> and the 'no' part leaves the 'match found' flag
>
> How ever, these all are *examples* on how you can get such or similar
> problems solved. It's on you to define rules, to find the required
> $Con{$fh}->{.....} flags in assp, to change the regex as needed and to
> write the code.
>
> The sub used in CorrectASSPcfg should be solid rock against crashes (in
> doubt: eval {} is a good friend)!
>
> Keep in mind: the header checks and body checks are done in a fixed order
> (assp_check_order.txt) - for example, it is useless to require a Con-flag
> in a headerRe, if this flag was not already processed (set) before by assp!
> be careful if you modify such flags - know what
> you do!
>
>
> >Not just that, but it helps me to understand ways that CorrectASSPcfg
> can be used.
>
> Reading the SSL GUI section, makes this really clear!
>
> CallBack to Verify ....
> Call to Configure .....
> ....
> ....
>
> To make it complete - the following subs in CorrectASSPcfg.pm will be
> called if they are available:
>
> from assp.pl:
>
> set - set hidden variables (prevents the requirement to modify assp.pl
> for hidden variable changes/settings)
> custom_mlog - do something with or because of log lines
> custom_reply - customize ASSP SMTP replies (literals, error explanations,
> .....)
> modMyHeader - check/modify X-ASSP-.... headers before they are added to
> the mail header
> translateReply - translate MTA SMTP reply codes (some MTA's provides bad
> reply codes)
> customAnalyze - called before a mail is parsed and processed by the
> analyzer
>
> from ASSP_AFC.pm:
>
> AFC_Executable_Detection - custome code to detect executable code in
> attachments, called if no code was found
>
>
> The following config parameters can point to subs in CorrectASSPcfg.pm:
>
> ASSP_ARCSelectCode
> ASSP_RSSSelectCode
> debugCode
>
>
> Thomas
>
>
> Von: "K Post" <nntp.p...@gmail.com>
> An: "ASSP development mailing list" <
> assp-test@lists.sourceforge.net>
> Datum: 14.07.2022 17:07
> Betreff: Re: [Assp-test] blockStrictDKIMRe -- also thoughts on
> DMARC rejects
> ------------------------------
>
>
>
> Outstanding.
> Heavy coding to implement my original blockStrictDKIMRe concept alone
> makes it not worth it. That, added to the can of worms that I'd be opening
> up by planting the seed to have other exceptions, is enough to put the nail
> in the coffin for the idea. Though if the strict regex for SPF failures
> for certain domains hasn't caused the kind of flood of request you're
> worried about, if at some point you think of an >easy< way to code the
> option, then maybe it should be reconsidered -- but again, only if it can
> be done without heavy coding on your part!!
>
> I do very much appreciate the discussion and thoughtful explanation. The
> relatively simple solution of putting code into CorrectASSPcfg is genius.
> Not just that, but it helps me to understand ways that CorrectASSPcfg can
> be used. I previously didn't know that we could get function results right
> in a regex from CorrectASSPcfg. Using a function offers so much
> flexibility and promise! I hope it doesn't sound pathetic, but that's
> exciting!
>
> A couple more questions if (or when) you have the time and energy for this:
>
>
> Why is the *FAIL bit in your example of
> ~<<<(\@.+\.docusign\.net|next domain|next
> domain|...|...)(?(?{&CorrectASSPcfg::myWantedDKIMCheck($fh,$+)})|(*FAIL))>>>~=>60
> I'm concerned about only matching (docusign.\net|otherdomains)(.*FAIL)
> I'm assuming you intended to have a period before the * Won't that
> match any header like:
>
> from: whate...@docusign.net
> subject: failure to complete submission
>
>
> The function example:
> sub CorrectASSPcfg::myWantedDKIMCheck {
> my ($fh,$match) = @_;
> my $this = ($fh && exists($main::Con{$fh})) ? $main::Con{$fh} :'';
> return unless $this;
> return unless $this->{isDKIM};
> return 1 if $this->{dkimresult} eq 'pass';
> return 1 if $this->{dkimverified} eq 'verified-OK';
> my $re = qr/
> domain1\.org
> | \.domain2\.org
> | user[^@]+?\@.+?\.domain3\.org
> /xis;
> return ($match !~ /$re/);
> }
> seems to return if there's no DKIM (return unless $this->{isDKIM};)
> wouldn't that not match the regex, so the 60 score wouldn't be applied?
> Part of my goal is to require DKIM signature for certain domain names, not
> only requiring valid DKIM.
>
> Thanks again
> Ken
>
>
>
>
> On Sat, Jul 9, 2022 at 4:53 AM Thomas Eckardt <
> *thomas.ecka...@thockar.com* <thomas.ecka...@thockar.com>> wrote:
> I'm sorry but the example
>
> \@.+\.docusign\.net(?{&CorrectASSPcfg::myWantedDKIMCheck($fh)})=>60
>
> should be better
>
> ~<<<(\@.+\.docusign\.net|next domain|next
> domain|...|...)(?(?{&CorrectASSPcfg::myWantedDKIMCheck($fh,$+)})|(*FAIL))>>>~=>60
>
>
> The first one does not fail if CorrectASSPcfg::myWantedDKIMCheck returns
> 0, The second provides $fh *and the matched string* to the sub
> CorrectASSPcfg::myWantedDKIMCheck.
>
> short example for CorrectASSPcfg::myWantedDKIMCheck
>
> sub CorrectASSPcfg::myWantedDKIMCheck {
> my ($fh,$match) = @_;
> my $this = ($fh && exists($main::Con{$fh})) ? $main::Con{$fh} :'';
> return unless $this;
> return unless $this->{isDKIM};
> return 1 if $this->{dkimresult} eq 'pass';
> return 1 if $this->{dkimverified} eq 'verified-OK';
> my $re = qr/
> domain1\.org
> | \.domain2\.org
> | user[^@]+?\@.+?\.domain3\.org
> /xis;
> return ($match !~ /$re/);
> }
>
>
> Thomas
>
>
> Von: "Thomas Eckardt" <*thomas.ecka...@thockar.com*
> <thomas.ecka...@thockar.com>>
> An: "ASSP development mailing list" <
> *assp-test@lists.sourceforge.net* <assp-test@lists.sourceforge.net>>
> Datum: 08.07.2022 16:53
> Betreff: Re: [Assp-test] blockStrictDKIMRe -- also thoughts on
> DMARC rejects
> ------------------------------
>
>
>
> If such a feature would be implemented, it will result in havy coding.
>
> >I want to outright block any message from @*.*docusign.net*
> <http://docusign.net/> that isn't signed or that has an invalid
> signature. I don't care if it's from a whitelisted email address, from an
> IP that's in the SPF record, and with a message body that is 100% great.
>
> You want not only to make the test domain based strict, you want to ignore
> flags like 'whitelisted' - that's ok - but if I would start to allow any
> flag exceptions, other users may want to have other or more flag exception.
>
> - noprocessing
> - whitelisted
> - spamlover
> - domain based scoring values
> - SMIME/PGP signed
> .....
>
> Yes, a great feature - but who would need it?
>
> The best way would be to create a level 1 plugin for this purpose. There
> you can check the dkim result, flags, ip's ... what ever you want - and
> based on your logic, you can block or pass the mail.
>
> But knowing (and thinking like) assp, will open other ways (solution
> workarounds) - for example.
>
> we assume the DKIM check is set to scoring - and the scoring value is 20
> points below the penalty limit.
>
> If a DKIM signature is invalid - assp scores.
> If the domain has ever sent a mail with a valid DKIM signature before (a
> DKIMCache entry is found), assp scores for DKIM if a mail without a DKIM
> signature from this domain is received.
>
> Now, if there was not added any other score (the mail is 100% ok, except
> DKIM) the mail will pass because the penalty limit is not reached. But you
> want to block the mail if the sender matches @*.*docusign.net*
> <http://docusign.net/>
>
> sender??? ... matches???... - assp has weighted regular expressions -
> like: bombSenderRe - where you can add or remove scoring points
> if you set there
> \@.+\.docusign\.net=>20
>
> all mails from those domains will get a penalty of 20 points, which is
> harmless if there is everything else ok with the mail
> if dkim fails, the penalty limit will be reached and the mail will be
> blocked
> this can be finetuned using :>NWLI
>
> You are also able to implement code in to the regex (for example to check
> for the DKIM result). This is much less complicated than writing a plugin.
> \@.+\.docusign\.net(?{&CorrectASSPcfg::myWantedDKIMCheck($fh)})=>60
> "score with 60 if the sender matches and the sub
> CorrectASSPcfg::myWantedDKIMCheck returned 1"
>
> Both examples should only show, that there are more ways to get wanted
> results in assp. If someone solved a similar problem using another way, it
> would be nice to hear, how this was done.
>
>
> Thomas
>
>
>
>
> Von: "K Post" <*nntp.p...@gmail.com* <nntp.p...@gmail.com>>
> An: "ASSP development mailing list" <
> *assp-test@lists.sourceforge.net* <assp-test@lists.sourceforge.net>>
> Datum: 07.07.2022 15:56
> Betreff: Re: [Assp-test] blockStrictDKIMRe -- also thoughts on
> DMARC rejects
> ------------------------------
>
>
>
> All of your points are clear, and the explanation is greatly appreciated.
> I now understand why it may be unwise to generally honor reject DMARC
> policy if we've overridden spf/dkim policy once we start manipulating
> results with ASSP. That makes sense.
>
> I still feel like a *blockStrictDKIMRe* type of new feature, where a
> failed OR missing dkim signature where the message matches the regex would
> be strictly blocked (just like we can do with blockstrictSPFRe for spf
> failures) would be helpful.
>
> For example (hopefully this is more illustrative of the desire), I want to
> outright block any message from @*.*docusign.net* <http://docusign.net/>
> that isn't signed or that has an invalid signature. I don't care if it's
> from a whitelisted email address, from an IP that's in the SPF record, and
> with a message body that is 100% great. If there's no DKIM signature or an
> invalid one for a message that matches the regex, reject the message (just
> like their DMARC policy says to do).
>
> Is there another way with current ASSP features to accomplish this only if
> a message matches this proposed regex?
>
> Ken
>
>
> On Fri, Jun 17, 2022 at 4:35 AM Thomas Eckardt <
> *thomas.ecka...@thockar.com* <thomas.ecka...@thockar.com>> wrote:
> >*Would you please consider adding a feature to do the same for a failed
> DKIM signature?*
>
> NO!
>
> Contrary to SPF, a DKIM signature has only two options : OK and FAIL -
> Based on the signature it self or based on a trusted forwarders
> authentication result (ARC).
> A DKIM signature has to be valid every time for any of the above reasons.
>
> > I score failed spf and score failed dkim, so DoDMARC is only scoring
> even though p=reject.
>
> What else makes sense?
> If SPF is scored and DKIM is scored and DMARC is score - AND the resulting
> score does'nt block the mail at the pealtybox, your settings are wrong!
>
>
> >*If DMARC says p=reject, why shouldn't assp outright honor that*,
> regardless of if we have spf / dkim failures set to only score?
>
> SPF has too many options to change/override the original result in assp
> (more or less strict, overwrite, skip ....), some these options also exists
> for DKIM.
> If we ignore/change/override .... sender policies for SPF and DKIM, it is
> not wise to honor the reject DMARC policy strictly.
>
> Thomas
>
>
>
>
> Von: "K Post" <*nntp.p...@gmail.com* <nntp.p...@gmail.com>>
> An: "ASSP development mailing list" <
> *assp-test@lists.sourceforge.net* <assp-test@lists.sourceforge.net>>
> Datum: 16.06.2022 19:28
> Betreff: [Assp-test] blockStrictDKIMRe -- also thoughts on DMARC
> rejects
> ------------------------------
>
>
>
> The ability to block failed SPF, instead of just scoring them, for delect
> regex matches has been a terrific feature of ASSP for a long time.
> (Block SPF Processing Regex* (blockstrictSPFRe) ) *Would you please
> consider adding a feature to do the same for a failed DKIM signature?*
> Outright blocking of a matching message that fails DKIM, regardless of the
> domain's DMARC settings. -- maybe that's not necessary if DoDMARC will
> honor =reject, see more below.
>
> Reasoning:
> I already score failed DKIM signatures, but I can't set that score too
> high because so many organizations still send messages through 3rd parties
> with invalid DKIM signatures. It really is incredible how many I see. But
> for frequently abused sender addresses (docusign for example), who are
> often spoofed but send otherwise unspammy content, I want to outright block
> if the DKIM signature fails. blockStrictSPFRe usually works because these
> bad DKIM sigs are on mails that also violate SPF rules, still though it
> would be helpful if I could also just say "if a specific regex is matched
> on an email with an invalid DKIM, reject the message"
>
> RELATED: DMARC p=reject should always reject if failed
> Docusign.net has a dmarc rule of p=reject. I want to honor that. The
> last scam that came in from them failed SPF and failed DKIM validation, but
> the message was from a whitelisted address.. DoDMARC says that the
> blocking will be the "most less aggressive" (least aggressive) and the
> published DMARC record. I score failed spf and score failed dkim, so
> DoDMARC is only scoring even though p=reject.
>
> Enable DMARC Check (DoDMARC)
> If enabled and ValidateSPF and DoDKIM are enabled and the sending domain
> has published a DMARC-record/policy, assp will act on the mail according to
> the senders DMARC-policy using the results of the SPF and DKIM check and
> validating the SPF/DKIM address/domain Identifier Alignment rules (RFC7489
> section 3). It is safe to leave this feature ON, it will not produce false
> positives! The blocking mode (block, monitor, score, testmode) is adapted
> from the most less aggressive setting of ValidateSPF and DoDKIM - and the
> published DMARC record ([p][sp]=[reject][quarantine]). Scoring is done
> using dmarcValencePB.
>
> * If DMARC says p=reject, why shouldn't assp outright honor that*,
> regardless of if we have spf / dkim failures set to only score?
>
> Thanks
> Ken
>
> _______________________________________________
> Assp-test mailing list
> *Assp-test@lists.sourceforge.net* <Assp-test@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/assp-test*
> <https://lists.sourceforge.net/lists/listinfo/assp-test>
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
> _______________________________________________
> Assp-test mailing list
> *Assp-test@lists.sourceforge.net* <Assp-test@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/assp-test*
> <https://lists.sourceforge.net/lists/listinfo/assp-test>
> _______________________________________________
> Assp-test mailing list
> *Assp-test@lists.sourceforge.net* <Assp-test@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/assp-test*
> <https://lists.sourceforge.net/lists/listinfo/assp-test>
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
> _______________________________________________
> Assp-test mailing list
> *Assp-test@lists.sourceforge.net* <Assp-test@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/assp-test*
> <https://lists.sourceforge.net/lists/listinfo/assp-test>
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
> _______________________________________________
> Assp-test mailing list
> *Assp-test@lists.sourceforge.net* <Assp-test@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/assp-test*
> <https://lists.sourceforge.net/lists/listinfo/assp-test>
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
