Comment on lint behavior

[joe a]

SpamAssassin 3.4.5 (2021-03-20)

Found that lint did not catch a "typo" I made in a "wildcard" email address in the 
directive "whitelist_auth"

Intended to enter an email address in the form"*@some.mail" but 
entered"^@some.mail" instead.  Lint did not complain.

Perhaps the caret character is valid and has some meaning in that context I'm 
not aware of, but thought I would mention it.

Re: Comment on lint behavior

[joe a]

On 10/30/2024 12:52:39, Bill Cole wrote:

On 2024-10-30 at 11:11:48 UTC-0400 (Wed, 30 Oct 2024 11:11:48 -0400)
joe a
is rumored to have said:


SpamAssassin 3.4.5 (2021-03-20)

Found that lint did not catch a "typo" I made in a "wildcard" email address in the 
directive "whitelist_auth"

Intended to enter an email address in the form"*@some.mail" but 
entered"^@some.mail" instead.  Lint did not complain.

SpamAssassin's lint function is performing as designed. You entered an entirely 
valid email address. The lint function in SA has no way to know that you 
actually meant to enter a wildcard.


Perhaps the caret character is valid and has some meaning in that context I'm 
not aware of, but thought I would mention it.

The caret is an entirely valid character for the local-part of an email 
address. It has NO special meaning in the context of an email address. See page 
12 of RFC5322, where ^ is explicitly included in the definition of legal 
characters in an email address.


Thanks for the information.

mailspike dot net Minus 1?

[joe a]

Noticed some obvious spam slipping in due in great part to this:

* -1.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
*  [209.85.166.199 listed in wl.mailspike.net]

Not a big deal for my low volume SOHO, but it's annoying.

Has that check become unreliable?  Sure, I can skip that check (I think) or 
alter the score, but any other thoughts?

Re: mailspike dot net Minus 1?

[joe a]

On 9/21/2024 14:06:28, Reindl Harald (privat) wrote:



Am 21.09.24 um 18:51 schrieb joe a:

Noticed some obvious spam slipping in due in great part to this:

* -1.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
*  [209.85.166.199 listed in wl.mailspike.net]

Not a big deal for my low volume SOHO, but it's annoying.

Has that check become unreliable?  Sure, I can skip that check (I 
think) or alter the score, but any other thoughts?


what makes you think a single rule is that important?

sometimes IPs on whitelists starting to send spam, somehtimes 
spamhosts are not on a blacklist until they are - so what's the fuss 
about?


100% clear spam won't survive just because of a single -1 rule


Here is a more complete list from a very similar message, received 
today.  I failed to report the last -1.0 when I posted earlier.


X-Spam-Report:
*  1.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
*  [score: 1.]
*  3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
*  [score: 1.]
* -0.9 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
*  [209.85.219.198 listed in wl.mailspike.net]
*  0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level
*  mail domains are different
*  0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
*  0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail)
*  1.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
*  provider
*  [lurramachile[at]att.net]
*  0.0 HTML_MESSAGE BODY: HTML included in message
* -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
*  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
*   valid
*  0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and
*  EnvelopeFrom freemail headers are different
* -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list
*   manager

Whitelist or BAYES?

[joe a]

Maybe I should not ask this, but . . .

A relatively innocuous member informational email from a local town Library 
(monthly) gets marked as spam as shown below.
The BAYES_99 and BAYES_999 values are something I am toying with for other 
reasons.  Seems odd these should hit either one of those tests.

So, on the one hand I can add them to whitelist and be done with it, or I can 
add
them to missed HAM for re-learning.

Which is the best approach?





X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on elmoid
X-Spam-Flag: YES
X-Spam-Level: *
X-Spam-Status: Yes, score=9.9 required=5.0 tests=BAYES_99,BAYES_999,
DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,
HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,SPF_HELO_NONE,SPF_SOFTFAIL,
T_KAM_HTML_FONT_INVALID autolearn=disabled version=3.4.5
X-Spam-Report:
*  4.1 BAYES_99 BODY: Bayes spam probability is 99 to 100%
*  [score: 1.]
*  5.0 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
*  [score: 1.]
*  0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
*  0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level
*  mail domains are different
*  0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail)
*  0.0 HTML_MESSAGE BODY: HTML included in message
* -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
* -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
*  author's domain
*  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
*   valid
*  0.0 T_KAM_HTML_FONT_INVALID Test for Invalidly Named or Formatted
*  Colors in HTML
* -0.0 DKIMWL_WL_MED DKIMwl.org - Medium trust sender
Received: . . .[re-dakt-ed]; Thu, 26 Sep 2024 08:05:44 -0400 (EDT)
X-Virus-Status: Clean

<<<

Re: Whitelist or BAYES?

[joe a]

On 9/30/2024 16:22:49, joe a wrote:

On 9/27/2024 04:05:51, Matus UHLAR - fantomas wrote:

On 26.09.24 10:27, joe a wrote:

Maybe I should not ask this, but . . .

A relatively innocuous member informational email from a local town 
Library (monthly) gets marked as spam as shown below.
The BAYES_99 and BAYES_999 values are something I am toying with for 
other reasons.  Seems odd these should hit either one of those tests.


So, on the one hand I can add them to whitelist and be done with it, 
or I can add

them to missed HAM for re-learning.

Which is the best approach?


so far, both. You may need to relearn multiple their (monthly) mails 
before it has effect.



X-Spam-Report:
*  4.1 BAYES_99 BODY: Bayes spam probability is 99 to 100%
*  [score: 1.]
*  5.0 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
*  [score: 1.]


You have raised BAYES_99 and BAYES_999 to huge values so I recommend 
to rethink that.


You some "don't because" examples?   Seems to me, off hand, that if 
it's 99% or 99.9% then a high value does no harm.  Perhaps half what I 
have would be sufficient though.


* -0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature

* -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
*  author's domain


you can safely welcomelist_from_dkim their mail address.

Can you expand on that a bit?  Did not know there was such an item.  
Is it obvious in the documentation?


I did find it clearly documented, eventually, but need to state 
whitelist rather than welcomelist not being at version 4.

Re: Whitelist or BAYES?

[joe a]

On 9/27/2024 04:05:51, Matus UHLAR - fantomas wrote:

On 26.09.24 10:27, joe a wrote:

Maybe I should not ask this, but . . .

A relatively innocuous member informational email from a local town 
Library (monthly) gets marked as spam as shown below.
The BAYES_99 and BAYES_999 values are something I am toying with for 
other reasons.  Seems odd these should hit either one of those tests.


So, on the one hand I can add them to whitelist and be done with it, 
or I can add

them to missed HAM for re-learning.

Which is the best approach?


so far, both. You may need to relearn multiple their (monthly) mails 
before it has effect.



X-Spam-Report:
*  4.1 BAYES_99 BODY: Bayes spam probability is 99 to 100%
*  [score: 1.]
*  5.0 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
*  [score: 1.]


You have raised BAYES_99 and BAYES_999 to huge values so I recommend 
to rethink that.


You some "don't because" examples?   Seems to me, off hand, that if it's 
99% or 99.9% then a high value does no harm.  Perhaps half what I have 
would be sufficient though.


* -0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature

* -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
*  author's domain


you can safely welcomelist_from_dkim their mail address.

Can you expand on that a bit?  Did not know there was such an item.  Is 
it obvious in the documentation?

OT - logrotate size parameter

[joe a]

This is OT, but perhaps someone here knows.

In the context of the logrotate conf file, what does the + sign indicate 
when used as a prefix size directive?


Example: "size +4096k"

Some conf files have it, some don't.  Man pages do not mention it AFAICT 
and the internet is rather seems to ignore it.

subscribe to blacklist for domains

[joe a]

I need to refresh my brain on using blacklists with SA, before looking 
more deeply into why this got through.


Today a email slipped through with a very low score that was clearly 
phishy.   A url in question, posing as another, hits no less that 6 
blacklists.  I was going to look at clamav that is in use here, as I had 
just been tuning that a bit and realized that that may be using a hammer 
to drive a screw.  so to speak.


Or are they passe these days?

Re: subscribe to blacklist for domains

[joe a]

On 8/12/2022 11:43 PM, Noel Butler wrote:

Why are you not blocking with blacklists at the border, ie: MTA.


I'm not familiar with how to do that or if it can be done.  Since SA 
offers this functionality, so did not even consider that. I'll look into it.


Given its 0 resources for your MTA, with anti spam checking on SA often 
using significant resources (depending on traffic/number of tests/rules 
etc), its best to stop it getting to SA in the first place.


SA also has this by-default list of domains that it never checks, for 
along time I have disagreed with this, we are the ones to decide who 
gets whitelisted not SA, not some paid third party, the option 
clear_uridnsbl_skip_domain  however prevents this, but then you have to 
locate and 0 all the general rulesets scores that are whitelists as well.




The configuration/usage of those lists causes me great frustration. 
Semi retirement and infrequent "tech stuff" may be partly to blame.

Re: subscribe to blacklist for domains

[joe a]

Ah, thanks for describing that.  I am somewhat more brain fogged than 
usual this morning, so am uncertain any of those would work in this 
configuration.  But I certainly need to look deeper. At least into my 
coffee mug.


This is a low volume system consisting of postfix, SA, clamav and 
fetchmail.


The mailserver (postfix) is not exposed to the internet, mail traffic is 
sent to it by "fetchmail", which itself goes out to several providers 
where mail accounts reside.


My first thought was, the postfix stuff would work, because . . . then I 
realized, I've not looked at those solutions for some time, if ever. So, 
I should stop here and look them over.


However, any real world "we did that" exists, please let me know.

joe a.

On 8/13/2022 9:52 AM, Bert Van de Poel wrote:
I think what Noel is referring to is Postfix configuration like this for 
example:
smtpd_recipient_restrictions = permit_mynetworks, 
permit_sasl_authenticated, reject_unauth_destination, reject_rbl_client 
zen.spamhaus.org, reject_rhsbl_reverse_client dbl.spamhaus.org, 
reject_rhsbl_helo dbl.spamhaus.org, reject_rhsbl_sender 
dbl.spamhaus.org, reject_non_fqdn_recipient, 
reject_unknown_recipient_domain


Notice the spamhaus links for different blocklist settings.

On 13/08/2022 15:38, joe a wrote:

On 8/12/2022 11:43 PM, Noel Butler wrote:

Why are you not blocking with blacklists at the border, ie: MTA.


I'm not familiar with how to do that or if it can be done.  Since SA 
offers this functionality, so did not even consider that. I'll look 
into it.


Given its 0 resources for your MTA, with anti spam checking on SA 
often using significant resources (depending on traffic/number of 
tests/rules etc), its best to stop it getting to SA in the first place.


SA also has this by-default list of domains that it never checks, for 
along time I have disagreed with this, we are the ones to decide who 
gets whitelisted not SA, not some paid third party, the option 
clear_uridnsbl_skip_domain  however prevents this, but then you have 
to locate and 0 all the general rulesets scores that are whitelists 
as well.




The configuration/usage of those lists causes me great frustration. 
Semi retirement and infrequent "tech stuff" may be partly to blame.

Re: subscribe to blacklist for domains

[joe a]

And, of course, I must edit my last reply:

On 8/13/2022 10:21 AM, joe a wrote:

My first thought was, the postfix stuff would work, because . . . 


My first thought was, the postfix stuff would NOT work, because . . .

Re: subscribe to blacklist for domains

[joe a]

On 8/13/2022 12:38 PM, Martin Gregorie wrote:
. . .
   
2) There's no mandatory need to REJECT spam. It has always been up to

the recipient to decide whether to return it to the sender or not.


Agreed in part.  I see returning SPAM to sender as an exercise in 
futility or perhaps further enabling.  But I do prefer labeling as SPAM 
to outright rejection in many cases.



3) It would be rather trivial to return spam to sender with a suitable
admonishment but I decided that its not worth my time to write such
a discriminator and maintain yet another set of rules about what gets
quarantined and what gets returned: better to quarantine it so
it can be analysed with the mk 1 eyeball.

Martin



To add my comment, returning SPAM, assuming it even reaches the original 
sender, may serve only to assure them of the effectiveness of their 
campaign to reach valid addresses. In effect "helping" them.


Opinions vary, of course.

Re: subscribe to blacklist for domains

[joe a]

I'll be sure to look this over well to see what I can use or adapt, thanks.


On 8/13/2022 11:04 AM, Reindl Harald wrote:



Am 13.08.22 um 16:21 schrieb joe a:
Ah, thanks for describing that.  I am somewhat more brain fogged than 
usual this morning, so am uncertain any of those would work in this 
configuration.  But I certainly need to look deeper. At least into my 
coffee mug.


This is a low volume system consisting of postfix, SA, clamav and 
fetchmail.


The mailserver (postfix) is not exposed to the internet, mail traffic 
is sent to it by "fetchmail", which itself goes out to several 
providers where mail accounts reside.


My first thought was, the postfix stuff would work, because . . . then 
I realized, I've not looked at those solutions for some time, if ever. 
So, I should stop here and look them over.


However, any real world "we did that" exists, please let me know


if 8 years in production is enough for you look below and keep in mind 
that this is for a inbound-only server and must not be applied to 
submission


postscreen_bare_newline_enable   = no
postscreen_bare_newline_action   = enforce
postscreen_pipelining_enable = no
postscreen_pipelining_action = enforce
postscreen_non_smtp_command_enable   = no
postscreen_non_smtp_command_action   = enforce

postscreen_dnsbl_min_ttl = 30s
postscreen_dnsbl_max_ttl = 30s
postscreen_dnsbl_threshold   = 8
postscreen_dnsbl_action  = enforce
postscreen_greet_action  = enforce
postscreen_greet_wait    = 10

postscreen_dnsbl_sites = dnsbl.sorbs.net=127.0.0.10*9
  dnsbl.sorbs.net=127.0.0.14*9
  zen.spamhaus.org=127.0.0.[10;11]*8
  dnsbl.sorbs.net=127.0.0.5*7
  zen.spamhaus.org=127.0.0.[4..7]*7
  b.barracudacentral.org=127.0.0.2*7
  zen.spamhaus.org=127.0.0.3*7
  dnsbl.inps.de=127.0.0.2*7
  hostkarma.junkemailfilter.com=127.0.0.2*4
  dnsbl.sorbs.net=127.0.0.7*4
  bl.spameatingmonkey.net=127.0.0.[2;3]*4
  dnsrbl.swinog.ch=127.0.0.3*4
  ix.dnsbl.manitu.net=127.0.0.2*4
  psbl.surriel.com=127.0.0.2*4
  bl.spamcop.net=127.0.0.2*4
  bl.mailspike.net=127.0.0.[10;11;12]*4
  bl.mailspike.net=127.0.0.2*4
  zen.spamhaus.org=127.0.0.2*3
  dnsbl.sorbs.net=127.0.0.6*3
  dnsbl.sorbs.net=127.0.0.8*2
  hostkarma.junkemailfilter.com=127.0.0.4*2
  dnsbl.sorbs.net=127.0.0.9*2
  dnsbl-1.uceprotect.net=127.0.0.2*2
  all.spamrats.com=127.0.0.38*2
  bl.nszones.com=127.0.0.[2;3]*1
  dnsbl-2.uceprotect.net=127.0.0.2*1
  dnsbl.sorbs.net=127.0.0.2*1
  dnsbl.sorbs.net=127.0.0.4*1
  dnsbl.sorbs.net=127.0.0.3*1
  hostkarma.junkemailfilter.com=127.0.1.2*1
  dnsbl.sorbs.net=127.0.0.15*1
  ips.backscatterer.org=127.0.0.2*1
  bl.nszones.com=127.0.0.5*-1
  wl.mailspike.net=127.0.0.[18;19;20]*-2
  hostkarma.junkemailfilter.com=127.0.0.1*-2
  ips.whitelisted.org=127.0.0.2*-2
  list.dnswl.org=127.0.[0..255].0*-2
  dnswl.inps.de=127.0.[0;1].[2..10]*-2
  list.dnswl.org=127.0.[0..255].1*-3
  list.dnswl.org=127.0.[0..255].2*-4
  list.dnswl.org=127.0.[0..255].3*-5

Re: subscribe to blacklist for domains

[joe a]

I am far from an anti SPAM expert, but:

On 8/13/2022 4:52 PM, Vincent Lefevre wrote:

On 2022-08-13 14:05:43 -0400, joe a wrote:

On 8/13/2022 12:38 PM, Martin Gregorie wrote:
. . .

2) There's no mandatory need to REJECT spam. It has always been up to
 the recipient to decide whether to return it to the sender or not.


Agreed in part.  I see returning SPAM to sender as an exercise in futility
or perhaps further enabling.  But I do prefer labeling as SPAM to outright
rejection in many cases.


Rejecting mail (instead of accepting it and dropping it) is useful
in case of false positives.


That may be so and of use to a legitimate sender that actually cares 
about such things.   A true SPAM'er could not care less.



3) It would be rather trivial to return spam to sender with a suitable
 admonishment but I decided that its not worth my time to write such
 a discriminator and maintain yet another set of rules about what gets
 quarantined and what gets returned: better to quarantine it so
 it can be analysed with the mk 1 eyeball.


To add my comment, returning SPAM, assuming it even reaches the original
sender, may serve only to assure them of the effectiveness of their campaign
to reach valid addresses. In effect "helping" them.


Well, if you don't reject the mail with the reason that the address
is invalid, the spammer could deduce that the address is valid
(at least potentially valid). By not rejecting spam, the spammer
could think that the spam arrived at its destination and would
validate the address.


Rejecting mail for an invalid recipient was not my concern.  In the case 
of an invalid email address is certainly proper to inform the sender of 
that fact.


I could even agree that informing senders of "false positives" is useful 
as well, but doing that via a "REJECT" would seem burdensome. 
REJECT-ing email that is flagged by one of the DNS RBL thingies still 
seems to me to be wasted effort and possibly counter productive.


Why waste your own system resources to help a scoundrel?  Drop them and 
be done.


joe a.

spamd config error

[joe a]

Noticed this line in /var/log/mail:

spamd[31188]: config: failed to parse line, skipping, in 
"/etc/mail/spamassassin/local.cf": Mail::SpamAssassin::Plugin::URIDNSBL


It seems to have started a few weeks ago and does not appear to be 
related to the date of any deliberate changes on my part.


Small home office system. My skills are a bit stale, so any assistance 
is gladly accepted.

Re: spamd config error

[joe a]

On 1/2/2023 2:49 PM, joe a wrote:

Noticed this line in /var/log/mail:

spamd[31188]: config: failed to parse line, skipping, in 
"/etc/mail/spamassassin/local.cf": Mail::SpamAssassin::Plugin::URIDNSBL


It seems to have started a few weeks ago and does not appear to be 
related to the date of any deliberate changes on my part.


Small home office system. My skills are a bit stale, so any assistance 
is gladly accepted.



It appears to be related to this line in local.cf:

Mail::SpamAssassin::Plugin::URIDNSBL

When I comment that out and restart spamd, the error no longer appears. 
 It appears OK, but perhaps my eyes are cheated by some spell?

Re: spamd config error

[joe a]

On 1/2/2023 4:01 PM, joe a wrote:

On 1/2/2023 2:49 PM, joe a wrote:

Noticed this line in /var/log/mail:

spamd[31188]: config: failed to parse line, skipping, in 
"/etc/mail/spamassassin/local.cf": Mail::SpamAssassin::Plugin::URIDNSBL


It seems to have started a few weeks ago and does not appear to be 
related to the date of any deliberate changes on my part.


Small home office system. My skills are a bit stale, so any assistance 
is gladly accepted.



It appears to be related to this line in local.cf:

Mail::SpamAssassin::Plugin::URIDNSBL

When I comment that out and restart spamd, the error no longer appears. 
  It appears OK, but perhaps my eyes are cheated by some spell?




Wow.  It appears you actually have to state "loadplugin".

A great way to start a new year.

Re: spamd config error

[joe a]

On 1/2/2023 4:27 PM, Bill Cole wrote:

On 2023-01-02 at 16:18:53 UTC-0500 (Mon, 2 Jan 2023 16:18:53 -0500)
joe a 
is rumored to have said:


On 1/2/2023 4:01 PM, joe a wrote:

On 1/2/2023 2:49 PM, joe a wrote:

Noticed this line in /var/log/mail:

spamd[31188]: config: failed to parse line, skipping, in 
"/etc/mail/spamassassin/local.cf": Mail::SpamAssassin::Plugin::URIDNSBL


It seems to have started a few weeks ago and does not appear to be 
related to the date of any deliberate changes on my part.


Small home office system. My skills are a bit stale, so any 
assistance is gladly accepted.



It appears to be related to this line in local.cf:

Mail::SpamAssassin::Plugin::URIDNSBL

When I comment that out and restart spamd, the error no longer 
appears.   It appears OK, but perhaps my eyes are cheated by some spell?




Wow.  It appears you actually have to state "loadplugin".


Yes. As documented. :)

You should not need to load that plugin in local.cf. It is loaded by 
default in init.pre.





Good to know.

I found the docs difficult to follow, initially, and just now, having 
not looked at them for a good while.  15 minute retraining window, you 
know.  Age is a cruel mistress.


Anyway, it is in init.pre as you say and I just confirmed it is not 
needed in local.cf.


Not implying any lack of faith you understand .

local rule exclude all domains except "my list of approved"

[joe a]

As an increasing amount of SPAM from "boutique" domains began slipping 
through, I resorted assuring they are marked as SPAM by adding custom 
rules when sufficiently annoyed.


The local rules take this form (thanks to whoever provided the 
"template" for this):


header __LOCAL_FROM_BE  From =~ /.\.beauty/i
meta LOCAL_BE (__LOCAL_FROM_BE)
score  LOCAL_BE 2
describe LOCAL_BE from beauty domain

Initially I thought it might be fun to create a "match everything except 
what I list in this rule", so will search the Camel book, to learn or 
refresh.


But, likely someone has already done this, or, there is a simpler way 
already devised, hence the post.


I do not want to block these outright, say at the firewall or Postfix 
level, just simply flagged as SPAM, as some of these might deserve 
review at least for entertainment value.

Re: local rule exclude all domains except "my list of approved"

[joe a]

On 1/5/2023 3:24 AM, Loren Wilton wrote:

You can simplify your rule code a little if you want:


header __LOCAL_FROM_BE  From =~ /.\.beauty/i
meta LOCAL_BE (__LOCAL_FROM_BE)
score  LOCAL_BE 2
describe LOCAL_BE from beauty domain


    to

header LOCAL_BE  From =~ /.\.beauty/i
score  LOCAL_BE 2
describe LOCAL_BE from beauty domain

The meta isn't really doing anything there, since it only has a single 
clause.
Metas are good when you want to combine the results of several matches 
with boolean logic.


You might also want to add a \b to the rule:

header LOCAL_BE  From =~ /.\.beauty\b/i

Without that the rule will match ".beauty", but also ".beautyrest".

Another thing you might want to consider is using "From:addr" rather 
than just "From". As it is, it will match ".beauty" both in the address 
and in the person's name description. So it would match:


    From: "janice.beautyfull" 

Maybe you want that, in wihich a bare "From" is fine.



Ah. Thanks.

Refused by block lists

[joe a]

Attempting to utilize the various block lists and find rejection 
messages in mail headers "blocked due to usage of an open resolver".


One of many things puzzling me at the moment is something found in the 
related Wiki that states "A: Third, if your email gateway is behind a 
firewall make sure that SpamAssassin is resolving the gateway to its 
external address."


I brazenly confess I have no idea how to check this (or what it means, 
in this context).


Figured I should sort out that puzzlement before attempting to install 
and configure "unbound" for example.

Re: Refused by block lists

[joe a]

On 1/6/2023 12:15 PM, Kevin A. McGrail wrote:

My interpretation is thus:

You have a firewall with a public IP and an private IP

You have a box with email behind that firewall.

When it talks to the world, it should do helo  that maps back to 
your Firewall's public IP not to a private RFC1918 address.


Regards,KAM


Make sense to me.

So I guess my real question is, how do I cause spamassassin to make it's 
query in that fashion?  Since the wiki stated it in a way that suggests 
it is a spamassassin feature, I presume to ask here and not look at the 
firewall or elsewhere.

Re: Refused by block lists

[joe a]

On 1/6/2023 12:49 PM, John Hardin wrote:

On Fri, 6 Jan 2023, joe a wrote:
. ..

I think you're getting distracted by the word "resolve" there... This 
sounds like a DNS issue.




Agree it is likely a DNS issue.  Apparently one I do not yet grasp.

Is there an online tool to which I can make a DNS query and have it 
display what it receives?   Trying to avoid having to packet sniff my 
outbound traffic.


I have captured DNS queries via the firewall log/filters, but would like 
to verify.

Re: Refused by block lists

[joe a]

On 1/7/2023 9:06 AM, Matus UHLAR - fantomas wrote:

On Fri, 6 Jan 2023, joe a wrote:
Attempting to utilize the various block lists and find rejection 
messages in mail headers "blocked due to usage of an open resolver".


On 06.01.23 09:49, John Hardin wrote:
Are you forwarding your SpamAssassin DNS queries to your ISP or (e.g.) 
Google?


Best practice is to set up a local, non-forwarding (potentially 
non-forwarding only for the DNSBL domains, see my email from a week or 
so back) DNS server for your MTA and SpamAssassin to use (potentially 
your entire local network as well, but that's not relevant to your 
question).


DNSBL providers generally don't like requests from public DNS servers 
as they aggregate a lot of requests from a lot of sources.


https://cwiki.apache.org/confluence/display/SPAMASSASSIN/DnsBlocklists

Q: My queries to a DNS-blocklist were blocked. What does this mean?
...

Resolving the block might be as simple as using your own non-forwarding 
caching nameserver


https://cwiki.apache.org/confluence/display/SPAMASSASSIN/CachingNameserver




Thanks.  I think I actually got unbound working but still was getting 
URIBL rejects from spamhaus.


I've disabled queries for now and will try again in a few days, thinking 
the "free use" limits may have been tripped.


That will give me some time to review how to disable specific checks, 
such as dnswl.org which caused a score of -5.0 for some obviously spammy 
stuff.

Re: Refused by block lists

[joe a]

On 1/7/2023 1:25 PM, Matus UHLAR - fantomas wrote:

On 1/7/2023 9:06 AM, Matus UHLAR - fantomas wrote:

https://cwiki.apache.org/confluence/display/SPAMASSASSIN/DnsBlocklists

Q: My queries to a DNS-blocklist were blocked. What does this mean?
...

Resolving the block might be as simple as using your own 
non-forwarding caching nameserver


https://cwiki.apache.org/confluence/display/SPAMASSASSIN/CachingNameserver


On 07.01.23 12:03, joe a wrote:
Thanks.  I think I actually got unbound working but still was getting 
URIBL rejects from spamhaus.


- do you actually use that unbound server? is 127.0.0.1 in 
/etc/resolv.conf?


Pretty sure.  Or, I was.  Ran various tests with unbound running and not 
running confirmed it was working, at least providing a response.  SA I 
told to use unbound via local.cf as well.


Right now unbound is disabled and DNS is via "my old way".


- doesn't unbound forward queries to other (isp, open) resolvers?



Not certain.  The docs/examples seemed a bit sparse suggesting it does 
and exceptions needed to be specified for spamhaus (for example) but did 
not provide examples of how to do that.   Some folks elsewhere seemed to 
suggest it would "just work".


Likely I need to learn how to configure it properly?

Re: Refused by block lists

[joe a]

On 1/7/2023 12:16 PM, Benny Pedersen wrote:

joe a skrev den 2023-01-07 18:03:


That will give me some time to review how to disable specific checks,
such as dnswl.org which caused a score of -5.0 for some obviously
spammy stuff.


please report spam https://www.dnswl.org/?page_id=17

especily for dnswl hi



I'll give it a try.  When I looked at dnswl.org the last updated comment 
seemed to be from 2017, so I kind of wrote it off as being unmaintained.


But, what do I know?

Re: Refused by block lists

[joe a]

On 1/8/2023 12:36 PM, Matus UHLAR - fantomas wrote:

On 07.01.23 12:03, joe a wrote:
Thanks.  I think I actually got unbound working but still was 
getting URIBL rejects from spamhaus.



On 1/7/2023 1:25 PM, Matus UHLAR - fantomas wrote:
- do you actually use that unbound server? is 127.0.0.1 in 
/etc/resolv.conf?


On 07.01.23 14:06, joe a wrote:
Pretty sure.  Or, I was.  Ran various tests with unbound running and 
not running confirmed it was working, at least providing a response. 


providing answer to my second question would spare you from guessing.


127.0.0.1 is not in /etc/resolv.conf.

I labor under the impression that telling unbound to accept query only 
on one IP and telling SA in local.cf "dns_server th.at.addr.ess" would 
cause it to use unbound.

Re: Refused by block lists

[joe a]

On 1/8/2023 2:08 PM, Martin Gregorie wrote:

On 07.01.23 14:06, joe a wrote:

Pretty sure.  Or, I was.  Ran various tests with unbound running
and
not running confirmed it was working, at least providing a
response.



Thats pretty simple to check, provided you've got Wireshark installed:
Fire it up and tell it to watch for DNS and/or blacklist lookup traffic
on the appropriate ports.

Then feed known spam to SA. Wireshark will show you if spam is causing
external lookup requests to be generated, where they are being sent, and
what replies are being received
  


Martin



Earlier I was going to do something like that, but at the 
firewall/router link to the cable modem.  I wanted to be sure the 
"source IP" was the site static IP.


A separate discussion uncovered I may have to register that IP with 
spamhaus.org.   Registered years ago and stopped using it.  Just now 
dawned that provider mergers cause my static IP's to change a few years 
back.


Almost every day I pass a "beef farmer" whose ponds and field teem with 
Canadian Geese.  Perhaps that should have been an omen?

excluding specific RBL checks

[joe a]

SA version 3.4.5

Gears are clashing, clutch is slipping, among other things.

Trying to exclude certain checks, via spamhouse services "by the book"

When placing these values in local.cf:

RCVD_IN_ZEN 0
RCVD_IN_XBL 0
RCVD_IN_PBL 0

"spamassassin --lint" complains. Yet SA starts without complaint and 
seems to not run those tests.


Placing "score" at the beginning of the line makes lint happy and SA 
seems to start fine and also does not run those tests.


So, one assumes it is a typo in the docs, or, one is expected to infer 
the "score" word.


Yet I still see this while "skip_rbl_checks 1" (in both above scenarios):

"RCVD_IN_ZEN_BLOCKED_OPENDNS RBL: ADMINISTRATOR NOTICE:"

Which suggests that one runs despite the directive or, I am using the 
wrong one.

Re: excluding specific RBL checks

[joe a]

On 1/8/2023 3:50 PM, joe a wrote:

SA version 3.4.5

Gears are clashing, clutch is slipping, among other things.

Trying to exclude certain checks, via spamhouse services "by the book"

When placing these values in local.cf:

RCVD_IN_ZEN 0
RCVD_IN_XBL 0
RCVD_IN_PBL 0

"spamassassin --lint" complains. Yet SA starts without complaint and 
seems to not run those tests.


Placing "score" at the beginning of the line makes lint happy and SA 
seems to start fine and also does not run those tests.


So, one assumes it is a typo in the docs, or, one is expected to infer 
the "score" word.


Yet I still see this while "skip_rbl_checks 1" (in both above scenarios):

"RCVD_IN_ZEN_BLOCKED_OPENDNS RBL: ADMINISTRATOR NOTICE:"

Which suggests that one runs despite the directive or, I am using the 
wrong one.





And the answer to the latter is "I had the wrong directive".  Which is 
obvious.  Now.

Re: excluding specific RBL checks

[joe a]

On 1/8/2023 4:00 PM, joe a wrote:

On 1/8/2023 3:50 PM, joe a wrote:

SA version 3.4.5

Gears are clashing, clutch is slipping, among other things.

Trying to exclude certain checks, via spamhouse services "by the book"

When placing these values in local.cf:

RCVD_IN_ZEN 0
RCVD_IN_XBL 0
RCVD_IN_PBL 0

"spamassassin --lint" complains. Yet SA starts without complaint and 
seems to not run those tests.


Placing "score" at the beginning of the line makes lint happy and SA 
seems to start fine and also does not run those tests.


So, one assumes it is a typo in the docs, or, one is expected to infer 
the "score" word.


Yet I still see this while "skip_rbl_checks 1" (in both above scenarios):

"RCVD_IN_ZEN_BLOCKED_OPENDNS RBL: ADMINISTRATOR NOTICE:"

Which suggests that one runs despite the directive or, I am using the 
wrong one.





And the answer to the latter is "I had the wrong directive".  Which is 
obvious.  Now.




Correcting myself, yet again, "score" needs to be specified, it seems, 
otherwise this is seen in /var/log/mail:


2023-01-08T15:00:42.854109-05:00 auxilary spamd[14937]: config: failed 
to parse line, skipping, in "/etc/mail/spamassassin/local.cf": RCVD_IN_ZEN 0
2023-01-08T15:00:42.854573-05:00 auxilary spamd[14937]: config: failed 
to parse line, skipping, in "/etc/mail/spamassassin/local.cf": RCVD_IN_XBL 0
2023-01-08T15:00:42.854908-05:00 auxilary spamd[14937]: config: failed 
to parse line, skipping, in "/etc/mail/spamassassin/local.cf": RCVD_IN_PBL 0


Contrary to some, there is value in following logs when making changes.
who'd have thought that.

Re: excluding specific RBL checks

[joe a]

On 1/8/2023 4:23 PM, Charles Sprickman wrote:

What did you end up with?


score RCVD_IN_ZEN_BLOCKED_OPENDNS 0

I am not certain if that stops the test or simply reporting of the 
message.  Looks like I will need to do some packet capture after all.



I have a bunch of zero rules for these yet still keep getting the "administrative 
notice" from sbl/zen.

The fact that those guys don't just send out a "yes, this is on by default in 
spamassassin, here is copy pasta to turn us off" email bugs me.

I've grown to this huge list and still get the warnings.

# remove spamhaus tests, they want us to pay
# need to include the first base rule or DNS still triggers but is ignored
score __RCVD_IN_ZEN 0


Is that a typo? There should be no underscore before RCVD, correct?


score RCVD_IN_SBL 0
score RCVD_IN_XBL 0
score RCVD_IN_PBL 0
score URIBL_SBL 0
score URIBL_CSS 0
score URIBL_SBL_A 0
score URIBL_CSS_A 0
score URIBL_DBL_SPAM 0
score URIBL_DBL_PHISH 0
score URIBL_DBL_MALWARE 0
score URIBL_DBL_BOTNETCC 0
score URIBL_DBL_ABUSE_SPAM 0
score URIBL_DBL_ABUSE_REDIR 0
score URIBL_DBL_ABUSE_PHISH 0
score URIBL_DBL_ABUSE_MALW 0
score URIBL_DBL_ABUSE_BOTCC 0

Until I can get around to updating I'm considering just nuking the actual tests 
from the ruleset.

Charles

Re: excluding specific RBL checks

[joe a]

On 1/8/2023 4:38 PM, Benny Pedersen wrote:

joe a skrev den 2023-01-08 21:50:

SA version 3.4.5

Gears are clashing, clutch is slipping, among other things.

Trying to exclude certain checks, via spamhouse services "by the book"


what book ?


The good one? Several places.  Most looked like cut and paste from each 
other.  Trying to find the exact place now and cannot. Saw it most 
recently on another list, where others happened to be having similar dns 
issues.



When placing these values in local.cf:

RCVD_IN_ZEN 0
RCVD_IN_XBL 0
RCVD_IN_PBL 0

"spamassassin --lint" complains. Yet SA starts without complaint and
seems to not run those tests.


you miss score in 3 lines ?


Yep.


Placing "score" at the beginning of the line makes lint happy and SA
seems to start fine and also does not run those tests.


so lint passed ?


Yes, with score.


So, one assumes it is a typo in the docs, or, one is expected to infer
the "score" word.


what docs ?

anythin on web is fake news, only valid docs is perldoc 
Mail::SpamAssassin::Conf



I only know of https://spamassassin.apache.org/full/3.4.x/doc/ which I 
though I was referencing.  Seems likely I just allowed myself to be 
misled, "chaff".



and all related plugins


Yet I still see this while "skip_rbl_checks 1" (in both above scenarios):


clear your config :)


"RCVD_IN_ZEN_BLOCKED_OPENDNS RBL: ADMINISTRATOR NOTICE:"
Which suggests that one runs despite the directive or, I am using the 
wrong one.


make /etc/resolv.conf only have nameserver 127.0.0.1 and you ether have 
bind, unbound, pdns-recursor as of your own choise


Certainly worth a try and much simpler that what I was trying.


still problems ?, lets hear them

Re: excluding specific RBL checks

[joe a]

On 1/8/2023 10:35 PM, Henrik K wrote:

On Sun, Jan 08, 2023 at 04:23:11PM -0500, Charles Sprickman wrote:

. . .
# remove spamhaus tests,. . .
score RCVD_IN_SBL 0
score RCVD_IN_XBL 0
score RCVD_IN_PBL 0
score URIBL_SBL 0
score URIBL_CSS 0
score URIBL_SBL_A 0. . . 

Much easier and reliable way:

dns_query_restriction deny spamhaus.org



Ah Hah!  Seems to work for me.  See? I CAN be taught!

joe a.

Re: excluding specific RBL checks

[joe a]

On 1/9/2023 3:55 AM, Matus UHLAR - fantomas wrote:
Until I can get around to updating I'm considering just nuking the 
actual tests from the ruleset.

Much easier and reliable way:

dns_query_restriction deny spamhaus.org



Charles Sprickman skrev den 2023-01-09 08:04:
Trying this on half the pair, I assume this hits all subdomains of 
spamhaus.org?


Never ran into that parameter in my searches for this.


On 09.01.23 09:26, Benny Pedersen wrote:

never read perldoc Mail::SpamAssassin::Conf ?


some people don't repeatedly read it thorough.

Henrik forgot this is pr domain, so fully domain including subdomain 
seen in "rndc querylog" in bind logs !


spamassassin -D -t spamtestmsg 2>&1 | less

dns_query_restriction deny dwl.dnswl.org list.dnswl.org
dns_query_restriction deny multi.uribl.com

imho score foo 0 is a bug


no, it's documented feature - rules with score 0 are not run.

However, joe a aka the OP should be more interested in finding out why 
are his DNS queries going through an open resolver and fixing the real 
issue.




Right you are.  It now appears resolved (cough, cough . . .).

Spamhaus site provided this quick test: "dig 2.0.0.127.zen.spamhaus.org 
+short" which with variant "dig @my.local.dns.serv 
2.0.0.127.zen.spamhaus.org +short", allowed me to pretty quickly sort it 
out.


A lot of cobwebs needed to be cleared out, but, seems to be working as 
advertised.


Thanks to all for their patience and suggestions.

joe a.

BAYES_00 BODY. Negative score?

[joe a]

Have some annoying SPAM that consistently shows a negative score on 
BAYES.  Is the default scoring or influenced by BAYES in some way?


*-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
*  [score: 0.]

SpamAssassin 3.4.5

Thanks for any pointers.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/13/2023 5:51 PM, Benny Pedersen wrote:

joe a skrev den 2023-02-13 23:42:

Have some annoying SPAM that consistently shows a negative score on
BAYES.  Is the default scoring or influenced by BAYES in some way?

*-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
*  [score: 0.]

SpamAssassin 3.4.5


time to upgrade imho :=)

or train bayes to know what is spam or not spam, if it fails turn off 
autolearn, make a burdon what is autolearned


in local.cf

bayes_auto_learn_threshold_nonspam n.nn (default: 0.1)
The score threshold below which a mail has to score, to be fed into 
SpamAssassin's learning systems automatically as a non-spam message.

bayes_auto_learn_threshold_spam n.nn (default: 12.0)
The score threshold above which a mail has to score, to be fed into 
SpamAssassin's learning systems automatically as a spam message.


i have changed scores on this 2 :)

now i dont need manuely training

above is a plugin that need to be enabled for this to work

remember to do a spamassassin --lint on changes of config files


So, what did you change them to, may I ask?  Not sure I really 
understand those limits.


In any case, I feed new SPAM and HAM into BAYES twice a day. via 
scripts, etc. so I really should have autolearn off, yes?


Maybe I need to retrain BAYES?  IIRC last time took "a long time".

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/13/2023 5:51 PM, Benny Pedersen wrote:

joe a skrev den 2023-02-13 23:42:

Have some annoying SPAM that consistently shows a negative score on
. . .


time to upgrade imho :=)
. . .


And, yes, I should upgrade.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/14/2023 2:56 AM, Matus UHLAR - fantomas wrote:

On 13.02.23 17:42, joe a wrote:
Have some annoying SPAM that consistently shows a negative score on 
BAYES.  Is the default scoring or influenced by BAYES in some way?


*-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
*  [score: 0.]


This indicates a mistrained database, which means you have trained too 
many spams or spam-like messages (commercial messages) as ham.


Proper training of spams should help. Just keep your spam (and 
optionally ham) corpora for retraining in case you would drop the database.


I also recommend to abstain from training commercial mail (notices from 
e-shops, companies you done business with etc) as ham, unless they 
generate BAYES_999 score and you want it lower.  I often train them as 
spam so those give uncertain BAYES_50 result.


Those mails resemble spam too much to be used for training.



All,

The term "proper training" has always seemed a bit problematic to me. 
That aside, experiencing an error trying attempting:


sa-learn -D --spam /var/mail/spamd/Cabinet.saved-spam

The last line shows:

***
Learned tokens from 0 message(s) (1 message(s) examined)
ERROR: the Bayes learn function returned an error, please re-run with -D 
for more information at /usr/bin/sa-learn line 500.

***

Which may be permissions related.  However, there seem to be some 
errors/warning at the beginning, starting with:


***
Feb 14 17:26:14.956 [2855] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::Razo r2 from 
@INC

Feb 14 17:26:14.959 [2855] dbg: razor2: razor2 is not available
Feb 14 17:26:14.959 [2855] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::SpamCop from @INC
plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/SpamCop.pm: 
lib/Mail/SpamAssassin/Plugin/SpamCop.pm: Permission denied at (eval 44) 
line 1.

***

While this also suggests a permissions issue the only place I find 
SpamCom.pm (even as root) is at: 
"/usr/lib/perl5/vendor_perl/5.26.1/Mail/SpamAssassin/Plugin/SpamCop.pm", 
which is not in the path sa-learn concocted when invoked.


Sorry if the formatting is weird or if this is useless information.

Re: BAYES_00 BODY. Negative score?

[joe a]

Please let this sit for a while, I've discovered a fundamental issue 
with my scheme of feeding messages to BAYES.  Unfortunately I was 
remiss, apparently, it setting up logging for some bits, so have no idea 
how long this has been failing.


Sorry for the clutter.

joe a.

On 2/14/2023 5:37 PM, joe a wrote:

On 2/14/2023 2:56 AM, Matus UHLAR - fantomas wrote:

On 13.02.23 17:42, joe a wrote:
Have some annoying SPAM that consistently shows a negative score on 
BAYES.  Is the default scoring or influenced by BAYES in some way?


*-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
*  [score: 0.]


This indicates a mistrained database, which means you have trained too 
many spams or spam-like messages (commercial messages) as ham.


Proper training of spams should help. Just keep your spam (and 
optionally ham) corpora for retraining in case you would drop the 
database.


I also recommend to abstain from training commercial mail (notices 
from e-shops, companies you done business with etc) as ham, unless 
they generate BAYES_999 score and you want it lower.  I often train 
them as spam so those give uncertain BAYES_50 result.


Those mails resemble spam too much to be used for training.



All,

The term "proper training" has always seemed a bit problematic to me. 
That aside, experiencing an error trying attempting:


sa-learn -D --spam /var/mail/spamd/Cabinet.saved-spam

The last line shows:

***
Learned tokens from 0 message(s) (1 message(s) examined)
ERROR: the Bayes learn function returned an error, please re-run with -D 
for more information at /usr/bin/sa-learn line 500.

***

Which may be permissions related.  However, there seem to be some 
errors/warning at the beginning, starting with:


***
Feb 14 17:26:14.956 [2855] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::Razo r2 from 
@INC

Feb 14 17:26:14.959 [2855] dbg: razor2: razor2 is not available
Feb 14 17:26:14.959 [2855] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::SpamCop from @INC
plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/SpamCop.pm: 
lib/Mail/SpamAssassin/Plugin/SpamCop.pm: Permission denied at (eval 44) 
line 1.

***

While this also suggests a permissions issue the only place I find 
SpamCom.pm (even as root) is at: 
"/usr/lib/perl5/vendor_perl/5.26.1/Mail/SpamAssassin/Plugin/SpamCop.pm", 
which is not in the path sa-learn concocted when invoked.


Sorry if the formatting is weird or if this is useless information.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/14/2023 6:09 PM, joe a wrote:
Please let this sit for a while, I've discovered a fundamental issue 
with my scheme of feeding messages to BAYES.  Unfortunately I was 
remiss, apparently, it setting up logging for some bits, so have no idea 
how long this has been failing.


Sorry for the clutter.

joe a.



Re-energized having recently heroically wrestled an elusive issue (to 
me) into surrender . . . we now turn to another issue.


Probably I need to retrain BAYES "From scratch".  I have a mess (years?) 
of stored sample emails that and be relearned.


I understand that sa-learn should be run as the same user as spamd, 
however I find it has always been run as root and when running as the 
spamassassin user results in errors, such as:


~su -c "sa-learn --spam /var/mail/spamd/Cabinet.Missed-SPAM" spamfilter

results in errors, starting with:

plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/SpamCop.pm: 
lib/Mail/SpamAssassin/Plugin/SpamCop.pm: Permission denied at (eval 44) 
line 1.


plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm: 
lib/Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm: Permission denied at 
(eval 45) line 1.


One might presume this to be a permissions issue (where would I get THAT 
idea?) but permissions to what?  As I cannot seem to find the items 
mentioned even as root.


Running with the -D option does produce more, after that list of 
permission denied items


Feb 16 15:55:30.884 [10384] dbg: config: warning: no description set for 
STOX_REPLY_TYPE_WITHOUT_QUOTES
Feb 16 15:55:30.884 [10384] dbg: config: warning: no description set for 
MSOE_MID_WRONG_CASE
Feb 16 15:55:30.884 [10384] dbg: config: warning: no description set for 
HELO_FRIEND
Feb 16 15:55:30.884 [10384] dbg: config: warning: no description set for 
STOX_AND_PRICE
Feb 16 15:55:30.884 [10384] dbg: config: warning: no description set for 
L_SPAM_TOOL_13
Feb 16 15:55:30.885 [10384] dbg: config: warning: no description set for 
FSL_FAKE_HOTMAIL_RVCD


Means something to someone I guess.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/16/2023 4:30 PM, Reindl Harald wrote:



Am 16.02.23 um 21:57 schrieb joe a:
I understand that sa-learn should be run as the same user as spamd, 
however I find it has always been run as root and when running as the 
spamassassin user results in errors, such as:


~su -c "sa-learn --spam /var/mail/spamd/Cabinet.Missed-SPAM" spamfilter

results in errors, starting with:

plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/SpamCop.pm: 
lib/Mail/SpamAssassin/Plugin/SpamCop.pm: Permission denied at (eval 
44) line 1.


plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm: 
lib/Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm: Permission denied 
at (eval 45) line 1.


One might presume this to be a permissions issue (where would I get 
THAT idea?) but permissions to what?  As I cannot seem to find the 
items mentioned even as root.


when you don't use proper packages and even can't update your mlocate 
database so that "locate SpamAssassin/Plugin/AutoLearnThreshold" that's 
hardly a SA topic


[root@mail-gw:~]$ rpm -q --file 
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm

spamassassin-3.4.6-5.fc36.x86_64

[root@mail-gw:~]$ rpm -q --file 
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/SpamCop.pm

spamassassin-3.4.6-5.fc36.x86_64


I have no idea what you refer to when you state "don't user proper 
packages".  "Proper" in what sense? A rhetorical question.


Mlocate is (was) not installed in this particular system but promises to 
be useful in the future, regardless of your intent.  "find" has always 
been my go to tool.  Such as it is.


Still it remains to be determined why root user can run sa-learn without 
error while another whose permissions are more constrained, cannot.


And that, regardless of root (!) cause, would seem to be an SA topic.

Re: BAYES_00 BODY. Negative score?

[joe a]

. . .


I have no idea what you refer to when you state "don't user proper 
packages".  "Proper" in what sense? A rhetorical question.


i have no idea how you installed SA but rpm packages or debs usually 
have correct permissions


Oh, of course.  I installed as root initially, being foolish perhaps, 
but did create a specific user "later" and adjusted permissions as 
needed.  Or, so I thought.


Mlocate is (was) not installed in this particular system but promises 
to be useful in the future, regardless of your intent.  "find" has 
always been my go to tool.  Such as it is.


Still it remains to be determined why root user can run sa-learn 
without error while another whose permissions are more constrained, 
cannot.


And that, regardless of root (!) cause, would seem to be an SA topic


because the file permissions are obviously wrong which isn't a SA topic 
- SA can't do anything when you mess your local permissions




Permissions are (almost) certainly the issue.  Now having the impressive 
locate/mlocate creature at my command, I might actually make progress.


Thanks for the help.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/16/2023 5:32 PM, hg user wrote:



On Thu, Feb 16, 2023 at 9:57 PM joe a <mailto:joea-li...@j4computers.com>> wrote:



plugin: failed to parse plugin (from @INC): Can't locate
Mail/SpamAssassin/Plugin/SpamCop.pm:
lib/Mail/SpamAssassin/Plugin/SpamCop.pm: Permission denied at (eval 44)
line 1.


root can do anything. a restricted user can't: it's only allowed to do 
what others allowed it.


it also runs with another environment, so it may miss PATHes or @INC 
directories.


That throws me a curve.  What is an @INC directory?  SA specific?
I do not find any with the locate command, but if the are an actual 
directory may need to escape the @ sign somehow.  \ does not seem to do it.



You should locate the SpamCop.pm file and list the owner and ACL.


This I have done, with no change, even to the point of starting using _R 
option at /usr/lib/perl5/vendor_perl/5.26.1/Mail



As user spamfilter run spamassassin with -D and see in the first lines 
if you have similar errors.


Done that.  It is impressively more verbose, but I did not detect any 
more errors.


Also check permission of /var/mail/spamd/Cabinet.Missed-SPAM. I had 
permission problems trying to sa-learn files owned by root.




That I found and fixed some time back.



Running with the -D option does produce more, after that list of
permission denied items

Feb 16 15:55:30.884 [10384] dbg: config: warning: no description set
for
STOX_REPLY_TYPE_WITHOUT_QUOTES


These are not permission errors but warnings about the rules having no 
text descriptions. It's ok.

Re: BAYES_00 BODY. Negative score?

[joe a]

. . .
it also runs with another environment, so it may miss PATHes or @INC 
directories.


That throws me a curve.  What is an @INC directory?  SA specific?
I do not find any with the locate command, but if the are an actual 
directory may need to escape the @ sign somehow.  \ does not seem to do it.




I being to see.  It is a perl thing.  I knew I should not have left that 
camel at the oasis.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/16/2023 8:28 PM, Matija Nalis wrote:


On Thu, Feb 16, 2023 at 05:34:37PM -0500, joe a wrote:

Oh, of course.  I installed as root initially, being foolish perhaps, but
did create a specific user "later" and adjusted permissions as needed.  Or,
so I thought.


well, installing as root (especially with restrictive umask) manually
(e.g. "make install" or "cpan" vs. "yum/rpm/dpkg") may often make
problems, even if you later switch to packages (you need to look not
only at final file permissions, but at directories leading up to it
too).

namei -l /path/to/file.pm is often helpful to quickly check ALL
permissions needed to access file (+x on directories is a must)


Permissions are (almost) certainly the issue.  Now having the impressive
locate/mlocate creature at my command, I might actually make progress.


I usually troubleshoot those (if log is insufficient) with:

strace -efile -o /tmp/sa.log spamassassin foobar

then look at /tmp/sa.log to see which open/stat/access returned -1 EPERM
or EACCES error.  Then check all path components for that file using
"namei -l" (or multiple "ls -ld"). Then try to su to that user and
"cat" that file manually.

If not regular DAC (chmod/chown) permissions, it might also be SELINUX
restrictions or more rarely ACL (getfacl(1)).



Well, I am in unfamiliar waters.

picking one error message as typical:

plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/iXhash2.pm: 
lib/Mail/SpamAssassin/Plugin/iXhash2.pm: Permission denied at (eval 
1746) line 1.


The file locations shown do not exist, as explicitly as shown.  What I 
find using "locate iXhash2.pm" is:


/usr/lib/perl5/vendor_perl/5.26.1/Mail/SpamAssassin/Plugin/iXhash2.pm
which the SA user can access, at least see via ll. The others I've 
checked are also visible, and directories are x (exccutable).


The sense I am getting is there is a perl file that contains these paths 
that is referred to as @INC.


I don't have the knowledge at this point to see if, somehow, root sees 
the files as shown in the error or if the path is somehow altered for 
the SA user.


Thanks for any guidance.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/17/2023 7:37 AM, Reindl Harald wrote:



Am 16.02.23 um 23:34 schrieb joe a:
I have no idea what you refer to when you state "don't user proper 
packages".  "Proper" in what sense? A rhetorical question.


i have no idea how you installed SA but rpm packages or debs usually 
have correct permissions


Oh, of course.  I installed as root initially, being foolish perhaps


you *must* install software as root because the service *must not* have 
write permissions to it's own binary files


but did create a specific user "later" and adjusted permissions as 
needed.  Or, so I thought


the real question was HOW DID YOU INSTALL it

from the first day i maintained production servers i learnt to build my 
own rpm packages - no matter if it's software written in C, PHP or Perl


why?

* because you get rid of leftover files over the years
* permissions are part of te package
* the package manager dectects many conflicts


One of the first things I learned when assembling things or attempting 
to learn something new, is to follow the instructions and only attempt 
to vary from them once you absolutely understood what your were doing. 
Or, suffer the consequences along with the (rare) accolades for 
improving a process.


That said, I would never "build my own rpm package" in this context.

This is almost entirely a "home/office" system that seems low traffic.

So, I installed postfix and spamassassin initially from the OS vendor 
supplied packages. Over the years I applied updates from outside the OS 
vendor channel, from packages from "authors" sites, as the versions 
diverged enough to be a concern.  There have been some OS updates as 
well and at least one transfer from one VM to another.


All this appears to be digression, to me, the issue, to me, seems to be 
why root sees the stuff in this @INC entity differently from how the SA 
user sees it.


With the insights and pointers gained in this thread, I hope to solve 
that sometime soon.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/17/2023 4:42 AM, Matus UHLAR - fantomas wrote:

On 16.02.23 15:57, joe a wrote:
Re-energized having recently heroically wrestled an elusive issue (to 
me) into surrender . . . we now turn to another issue.


Probably I need to retrain BAYES "From scratch".  I have a mess 
(years?) of stored sample emails that and be relearned.


I understand that sa-learn should be run as the same user as spamd, 
however I find it has always been run as root and when running as the 
spamassassin user results in errors, such as:


~su -c "sa-learn --spam /var/mail/spamd/Cabinet.Missed-SPAM" spamfilter

results in errors, starting with:

plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/SpamCop.pm: 
lib/Mail/SpamAssassin/Plugin/SpamCop.pm: Permission denied at (eval 
44) line 1.


plugin: failed to parse plugin (from @INC): Can't locate 
Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm: 
lib/Mail/SpamAssassin/Plugin/AutoLearnThreshold.pm: Permission denied 
at (eval 45) line 1.


try first changing current working directory into one readable by user 
"spamfilter", perhaps root (/).




Could it have been that simple?

Yes, apparently it was.

Many thanks.

joe a.

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/17/2023 11:44 AM, Martin Gregorie wrote:

On Fri, 2023-02-17 at 10:54 -0500, joe a wrote:


Could it have been that simple?


If, like myself, you find reference books useful, you may want to get a
copy of "Linux in a Nutshell" - an O'Reilly book.

It tends to assume you know at least one other OS fairly well, is well
organised and concise. I've also found "Debian Reference"

  http://www.debian.org/doc/manuals/debian-reference/

useful for most flavours of Linux (I use Fedora and Raspbian)

Martin



There was also a "Unix in a Nutshell".  I found it amusing, in my 
NetWare days, to have a copy on my desk and offer it to the Unix-oids 
that meanered in from time to time,  that liked to scoff at "security by 
obscurity" and those "Puny PC's you call Servers".  (That from folks 
that swore sendmail was forever king and operated the email server as an 
open relay).


A bit of an issue when I offered that the book should be called "Nuts, 
in a Unix Shell". . . Ah, the memories . . .

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/17/2023 3:25 PM, joe a wrote:

Did a simple test today sending an email from a gmail account to two 
email accounts on my system.   The only difference was the email 
address, both were on the same "To:" line in the composed messages.


They receive wildly different BAYES scores.
--
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on myserver
X-Spam-Level: *
X-Spam-Status: No, score=1.1 required=4.9 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE,
IXHASH_X1,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_SOFTFAIL
autolearn=disabled version=3.4.5
X-Spam-Report:
* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
*  [score: 0.0002]
--

X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on myserver
X-Spam-Flag: YES
X-Spam-Level: *
X-Spam-Status: Yes, score=5.2 required=4.9 tests=BAYES_20,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE,
IXHASH_X1,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_SOFTFAIL
autolearn=disabled version=3.4.5
X-Spam-Report:
*  2.2 BAYES_20 BODY: Bayes spam probability is 5 to 20%
*  [score: 0.0881]
--

Just another sign of BAYES wackiness? More evidence of need for rebuild?

Re: BAYES_00 BODY. Negative score?

[joe a]

On 2/17/2023 10:41 PM, Loren Wilton wrote:

They receive wildly different BAYES scores.
* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
*  [score: 0.0002]
*  2.2 BAYES_20 BODY: Bayes spam probability is 5 to 20%
*  [score: 0.0881]


This looks like you have per-user Bayes databases, and the messaage type 
has been trained differently in each.


Also, it looks like there are per-user rules, since BAYES_50 has a 
normal score of 0.2, and there is no reason BAYES_20 (indicating much 
less spammy) should have a score of 2.2.




Per-user is not setup.

This morning I sent the message again, with users reversed in the TO: 
field and the scores are identical.  This may prove nothing as I 
thoughtlessly added the high score message to my "HAM" folder and it was 
processed.


While the scores are identical the X-Spam-Report lists them in different 
order, while X-Spam-Status shows them identically, "RCVD_IN_MSPIKE_H2 
RBL" being listed near the top in one and near the bottom in the other.


Perhaps that is meaningless, but it pings my curiosity.

BAYES scores

[joe a]

Curious as to why these scores, apparently "stock" are what they are. 
I'd expect BAYES_999 BODY to count more than BAYES_99 BODY.


Noted in a header this morning:

*  3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
*  [score: 1.]
*  0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
*  [score: 1.]

Was this discussed recently?  I added a local score to mollify my sense 
of propriety.

Re: BAYES scores

[joe a]

On 2/28/2023 12:05 PM, Jeff Mincy wrote:

  > From: joe a 
  > Date: Tue, 28 Feb 2023 11:37:34 -0500
  >
  > Curious as to why these scores, apparently "stock" are what they are.
  > I'd expect BAYES_999 BODY to count more than BAYES_99 BODY.
  >
  > Noted in a header this morning:
  >
  > *  3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
  > *  [score: 1.]
  > *  0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
  > *  [score: 1.]
  >
  > Was this discussed recently?  I added a local score to mollify my sense
  > of propriety.

Those two rules overlap.   A message with bayes >= 99.9% hits both
rules.   BAYES_99 ends at 1.00 not .999.
-jeff



I get that they overlap.  I guess my thinker gets in a knot wondering 
why there is so little weight given to the more certain determination.


In my narrow view, anything that is 99.9% certain is probably worth a 5 
on it's own.  Or, at least should when, summed with BAYES_99, equal 5. 
As that is what the default "SPAM flag" is.


Appears more experienced or thoughtful persons think otherwise.

Yes, it did snow heavily overnight.  Yes, I am looking for excuses not 
to visit that issue.

when whitelisting, do what with marked SPAM?

[joe a]

Low volume home office user and system.

Occasionally when first dealing with a new entity, their correspondence 
gets flagged as SPAM.


When I whitelist these, what should be done with those messages that 
might remain in "flagged SPAM" or "Missed SPAM"?, thinking along lines 
of keeping BAYES "clean and sharp".  So to speak.


Leave as is?  Delete and re learn?

Re: when whitelisting, do what with marked SPAM?

[joe a]

On 11/14/2023 20:48:27, John Hardin wrote:

On Tue, 14 Nov 2023, joe a wrote:


Low volume home office user and system.

Occasionally when first dealing with a new entity, their 
correspondence gets flagged as SPAM.


When I whitelist these, what should be done with those messages that 
might remain in "flagged SPAM" or "Missed SPAM"?, thinking along lines 
of keeping BAYES "clean and sharp".  So to speak.


Leave as is?  Delete and re learn?


For a low volume home office user, I would simply NOT autolearn. Set up 
a hambox and a spambox and manually feed them and train from them.





I have autolearn off and have a spam and ham folder set up and "relearn" 
twice daily.

Re: when whitelisting, do what with marked SPAM?

[joe a]

On 11/14/2023 13:46:11, Matus UHLAR - fantomas wrote:

On 14.11.23 13:05, joe a wrote:

Low volume home office user and system.

Occasionally when first dealing with a new entity, their 
correspondence gets flagged as SPAM.


When I whitelist these, what should be done with those messages that 
might remain in "flagged SPAM" or "Missed SPAM"?, thinking along lines 
of keeping BAYES "clean and sharp".  So to speak.


Leave as is?  Delete and re learn?


Simply relearn FPs. Unless you have huge misclassification issue, 
learning as few mail as one should fix BAYES issues.




Move previously tagged SPAM into HAM folder and "relearn"?

Bayes "corpus" - how old?

[joe a]

Advisable to "prune" Bayes data based on age?

While cleaning up recent Ham/Spam, found my "saved SPAM" goes back to 
2013.


Why that's over . . . wait, I need to take off my socks . . .

So, how old is "too old".  For saved SPAM?

Re: Bayes "corpus" - how old?

[joe a]

On 1/30/2024 10:58:52, Matus UHLAR - fantomas wrote:

On 30.01.24 09:59, joe a wrote:

Advisable to "prune" Bayes data based on age?

While cleaning up recent Ham/Spam, found my "saved SPAM" goes back to 
2013.


Why that's over . . . wait, I need to take off my socks . . .

So, how old is "too old".  For saved SPAM?



I did retrain on old spam a few times and it was working fine.
Depends on how much mail you have:

0.000  0   7542  0  non-token data: nspam
0.000  0  80869  0  non-token data: nham
0.000  0 996032  0  non-token data: ntokens
0.000  0 1172945918  0  non-token data: oldest atime

so, even old spam mey be fine. You however need much of ham to train 
otherwise everything starts looking like spam.




Recently missed spam has increased a bit, so I was dropping it into 
"missed spam" and went poking through marked spam and found lots of 
"missed ham".Which triggered my pondering.

Re: Fake paypal email triggers -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM

[joe a]

On 2/6/2025 10:38:01, Bill Cole wrote:

On 2025-02-06 at 09:59:14 UTC-0500 (Thu, 6 Feb 2025 14:59:14 +)
Niamh Holding 
is rumored to have said:


Hello Giovanni,

Thursday, January 30, 2025, 2:28:18 PM, you wrote:

gpi> Paypal[.]com has been removed from default WL in November 
(https://github.com/apache/spamassassin/commit/76906e0c7c064391bf832b3eb885ae74aed6c8b5)

gpi> With updated rules USER_IN_DEF_DKIM_WL should not hit.

Though paypal.de still hits this rule as of yesterday.



Fixed. Should be in the distributed rules in a few days:

# svn commit -m "remove Paypal ccTLD domains from welcomelist"
Authentication realm:  ASF Committers
Password for 'billcole': ***

Sending    60_welcomelist_dkim.cf
Sending    60_welcomelist_spf.cf
Transmitting file data ..done
Committing transaction...
Committed revision 1923613.


FYI, I also hit this morning and am in USA (no further comment) with 
default for sa-update, which shows nothing to change.


(sorry for the off list reply earlier)

Re: Fake paypal email triggers -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM

[joe a]

On 1/29/2025 07:28:13, Greg Troxel wrote:

Niamh Holding writes:


Given the From: address can be so easily faked is a rule testing its validity a 
great idea?

This seems tricky to figure out.

The message's routing is obviously very sketchy.

But, it also appears that spamassassin has validated the DKIM signature
from paypal.com.  So the key question is whether

   - 1) this email was emitted from paypal's mail system
   - 2) paypal's DKIM signing key is compromised
   - 3) spamassassin is misparsing DKIM
   - 4) something else

I would take the message and run it through SA with -D -t.
I am guessing we are in case 1.

To be clear: if this is case 1, then it is not true that "the From:
address [is] faked".


If paypal is emitting user-generated content with DKIM signatures, then
they should be summarily removed from DKIM WL.  The point of those WL
entries are to cover mail that is really from those companies, believed
to be essentially never spam.

Also there is DKIMWL at high, and if you have a message with a valid
DKIM signature reporting it there as well would be good.


FWIW, I have seen these in several messages of this type:

. . .
* -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM
. . .

*  0.0 POSSIBLE_PAYPAL_PHISH_02 Claims to be from paypal but not
*  processed by any paypal MTA
*  1.0 POSSIBLE_PAYPAL_PHISH_03 Claims to be from paypal, sent to
*  Microsoft365 domain - likely fraud if you don't use MSFT365!