I have just gone through a few logs of expiration an message testing, and
have some more questions.
First of all, with the expiry, there are two stages which take a long time.
On my slow (450MHz, Windows) system once it has got to this line:
debug: bayes: expiry max exponent: 9
it takes seven minutes before it moves on.
=============================
debug: bayes: expiry check keep size, 0.75 * max: 375000
debug: bayes: token count: 498176, final goal reduction size: 123176
debug: bayes: First pass? Current: 1116929366, Last: 1116896823, atime:
5529600, count: 5721, newdelta: 256826, ratio: 21.5305016605489, period:
43200
debug: bayes: Can't use estimation method for expiry, something fishy,
calculating optimal atime delta (first pass)
debug: bayes: expiry max exponent: 9
debug: bayes: atime token reduction
debug: bayes: ======== ===============
debug: bayes: 43200 486091
debug: bayes: 86400 475937
debug: bayes: 172800 462174
debug: bayes: 345600 437604
debug: bayes: 691200 393656
debug: bayes: 1382400 330824
debug: bayes: 2764800 191807
debug: bayes: 5529600 1729
debug: bayes: 11059200 0
debug: bayes: 22118400 0
debug: bayes: First pass decided on 5529600 for atime delta
============================
What exactly is it doing at this stage? Why does it take so long, and is
there a way to reduce the time it takes to do this?
Again on my slow system it then takes a further 10 minutes to carry out
expiration.
============================
debug: refresh: 3096 refresh F:/DOCUME~1/ADMINI~1/SPAMAS~1/bayes.lock
debug: bayes: 3096 untie-ing
debug: bayes: 3096 untie-ing db_toks
debug: bayes: 3096 untie-ing db_seen
debug: bayes: files locked, now unlocking lock
debug: unlock: 3096 unlink F:/DOCUME~1/ADMINI~1/SPAMAS~1/bayes.lock
debug: Syncing complete.
============================
There's probably little i can do to speed this part up.
Now looking at a log of a message being scanned, it seems to check all
relays to see which ones are trusted.
score ALL_TRUSTED 0.0 0.0 0.0 0.0
============================
debug: metadata: X-Spam-Relays-Trusted: [ ip=127.0.0.1 rdns= helo=
by=arkbb.co.uk ident= envfrom= intl=0 id= auth= ]
debug: metadata: X-Spam-Relays-Untrusted: [ ip=80.168.70.64
rdns=mx4.mail.uk.clara.net helo=mx4.mail.uk.clara.net by=server. ident=
envfrom= intl=0 id=M2005052411250114443 auth= ] [ ip=213.47.158.114
rdns=chello213047158114.22.11.vie.surfer.at helo=autochair.co.uk
by=mx4.mail.uk.clara.net ident= envfrom= intl=0 id=1DaWas-0009VL-Bh auth= ]
============================
Is there any reason for it to continue to carry out these checks if all
trusted score set to 0 in my local.cf file?
I've noticed that SpamAssassin adds up the score as it goes along. If all
the uri tests hit, my scoring means that those emails are deleted
automatically my my mailserver as it has reached a high enough score.
However SA continues to check all rules before returning the result. Is it
possible to get it to stop tests once a specific score has been reached?
If this is possible, then is it possible to change the order in which SA
carries out the tests so i can put the uribl tests at the beginning, and
avoid having to carry out all of the others if all the uribl tests hit?
The majority of the emails that i have coming through my server and
therefore through SpamAssassin are actually downloaded from third party pop3
accounts or webmail accounts. Our system downloads emails from hotmail,
gmail, yahoo accounts as well as pop3 accounts and then redelivers to a
local account. All incoming emails (including the ones mentioned above) are
fed through a smtp proxy which deletes viruses.
Therefore all emails coming into our server come through either one or two
proxies before reaching SA.
The example below is actually an old email address which is automatically
forwarded to a local email address.
===========================
Received: from [127.0.0.1] by arkbb.co.uk with SMTP (HELO server.)
(ArGoSoft Mail Server Pro for WinNT/2000/XP, Version 1.8 (1.8.7.8)); Tue,
24 May 2005 11:25:15 +0100
Received: from mx4.mail.uk.clara.net ([80.168.70.64])
by server. (NAVGW 2.5.2.12) with SMTP id M2005052411250114443
for <[EMAIL PROTECTED]>; Tue, 24 May 2005 11:25:01 +0100
Received: from chello213047158114.22.11.vie.surfer.at ([213.47.158.114]
helo=autochair.co.uk)
by mx4.mail.uk.clara.net with smtp (Exim 4.46)
id 1DaWas-0009VL-Bh
for [EMAIL PROTECTED]; Tue, 24 May 2005 11:25:03 +0100
Received: from 254.225.74.17 by smtp.atta.cl;
Tue, 24 May 2005 10:32:24 +0000
==========================
>From these headers you can see that
254.225.74.17 sent it to smtp.atta.cl
213.47.158.114 sent it to mx4.mail.uk.clara.net
this was then forwarded to my server av proxy:
80.168.70.64 sent it to server
then to my mailserver which calls spamassassin:
127.0.0.1 sent it to arkbb.co.uk
Is there any way to do some kind of spf test, in this kind of case where
most emails are going through one or two proxies before reaching SA? SA
should be checking the spf record for atta.cl for the ip address
213.47.158.114. Is there a way to tell SA that this is what it should be
doing?
It looks like SA is detecting something funny itself, from the log below:
===========================
debug: X-Envelope-From header found after 1 or more Received lines, cannot
trust envelope-from
debug: Return-Path header found after 1 or more Received lines, cannot trust
envelope-from
debug: Running tests for priority: 0
debug: running header regexp tests; score so far=0
debug: registering glue method for check_for_spf_fail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f2010))
debug: SPF: relayed through one or more trusted relays, cannot use
header-based Envelope-From, skipping
debug: registering glue method for check_for_spf_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f2010))
debug: all '*From' addrs: [EMAIL PROTECTED]
debug: all '*To' addrs: [EMAIL PROTECTED] [EMAIL PROTECTED]
debug: registering glue method for check_hashcash_value
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x25dfa90))
debug: registering glue method for check_for_spf_helo_fail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f2010))
debug: SPF: checking HELO (helo=mx4.mail.uk.clara.net, ip=80.168.70.64)
debug: SPF: trimmed HELO down to 'clara.net'
debug: SPF: cannot load or create Mail::SPF::Query module
debug: registering glue method for check_for_spf_helo_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f2010))
debug: registering glue method for check_hashcash_double_spend
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x25dfa90))
debug: registering glue method for check_for_spf_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f2010))
debug: registering glue method for check_for_spf_helo_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f2010))
===========================
Why can't it trust the X-Envelope-From? If it can't trust this then what
domain will it perform a spf check on?
Regarding the SPF query, was it going to check with atta.cl that
80.168.70.64 was allowed to send? As i have already said this would be wrong
as i need it to check the sender before this due to the email forwarding
taking place.
Having checked another log where all the email forwarding is done internally
it seems to use the correct ip address. In the example before, an email has
come in to a pop3 account on tinyonline.co.uk. This has been downloaded with
a program which appears to call itself exchange-pop3-connector.com, pumped
through an AV proxy and into our mailserver.
An SPF check should check with the domain "alliedworldwide.com" whether the
ip address "210.180.27.23" is authorised to send from that domain... this
seems to be correct from the log (ie skipping the proxies and finding the
correct ip to check for).
===========================
debug: IP is reserved, not looking up PTR: 127.0.0.1
debug: received-header: parsed as [ ip=127.0.0.1 rdns= helo= by=arkbb.co.uk
ident= envfrom= intl=0 id= auth= ]
debug: received-header: parsed as [ ip=127.0.0.1
rdns=exchange-pop3-connector.com helo=exchange-pop3-connector.com by=server.
ident= envfrom= intl=0 id=M2005052412524512915 auth= ]
debug: received-header: parsed as [ ip=210.180.27.23 rdns=!210.180.27.23!
helo=!210.180.27.23! by=mk-cpfrontend.uk.tiscali.com ident= envfrom= intl=0
id=427BE49800F3211A auth= ]
debug: received-header: unknown format: from datafast.net.au (HELO
homewrecker) by mailhub.datafast.net.au with SMTP; Tue, 24 May 2005 04:44:00
-0800
debug: looking up A records for 'arkbb.co.uk'
debug: A records for 'arkbb.co.uk': 81.104.195.141
debug: received-header: 'from' 127.0.0.1 has reserved IP
debug: looking up A records for 'arkbb.co.uk'
debug: A records for 'arkbb.co.uk': 81.104.195.141
debug: received-header: 'by' arkbb.co.uk has public IP 81.104.195.141
debug: received-header: relay 127.0.0.1 trusted? yes internal? no
debug: received-header: 'from' 127.0.0.1 has reserved IP
debug: looking up A records for 'server.'
debug: A records for 'server.': 194.168.4.220
debug: received-header: 'by' server. has public IP 194.168.4.220
debug: received-header: relay 127.0.0.1 trusted? yes internal? no
debug: looking up A records for 'mk-cpfrontend.uk.tiscali.com'
debug: A records for 'mk-cpfrontend.uk.tiscali.com':
debug: received-header: relay 210.180.27.23 trusted? no internal? no
debug: metadata: X-Spam-Relays-Trusted: [ ip=127.0.0.1 rdns= helo=
by=arkbb.co.uk ident= envfrom= intl=0 id= auth= ] [ ip=127.0.0.1
rdns=exchange-pop3-connector.com helo=exchange-pop3-connector.com by=server.
ident= envfrom= intl=0 id=M2005052412524512915 auth= ]
debug: metadata: X-Spam-Relays-Untrusted: [ ip=210.180.27.23
rdns=!210.180.27.23! helo=!210.180.27.23! by=mk-cpfrontend.uk.tiscali.com
ident= envfrom= intl=0 id=427BE49800F3211A auth= ]
debug: ---- MIME PARSER START ----
debug: main message type: text/plain
debug: parsing normal part
debug: added part, type: text/plain
debug: ---- MIME PARSER END ----
debug: decoding: no encoding detected
debug: Loading languages file...
debug: Language possibly: en,sco
debug: metadata: X-Languages: en sco
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x2273320)
implements 'parsed_metadata'
debug: uri found: http://ensconce.greatpneumatic.com/ju30/
debug: URIDNSBL: domains to query: greatpneumatic.com
debug: X-Envelope-From header found after 1 or more Received lines, cannot
trust envelope-from
debug: Return-Path header found after 1 or more Received lines, cannot trust
envelope-from
debug: Running tests for priority: 0
debug: running header regexp tests; score so far=0
debug: registering glue method for check_for_spf_fail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f03bc))
debug: SPF: relayed through one or more trusted relays, cannot use
header-based Envelope-From, skipping
debug: registering glue method for check_for_spf_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f03bc))
debug: all '*From' addrs: [EMAIL PROTECTED]
[EMAIL PROTECTED]
debug: all '*To' addrs: [EMAIL PROTECTED]
[EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] [EMAIL PROTECTED]
debug: registering glue method for check_hashcash_value
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x25dedf0))
debug: registering glue method for check_for_spf_helo_fail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f03bc))
debug: SPF: checking HELO (helo=!210.180.27.23!, ip=210.180.27.23)
debug: SPF: trimmed HELO down to '27.23!'
debug: SPF: cannot load or create Mail::SPF::Query module
debug: registering glue method for check_for_spf_helo_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f03bc))
debug: registering glue method for check_hashcash_double_spend
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x25dedf0))
debug: registering glue method for check_for_spf_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f03bc))
debug: registering glue method for check_for_spf_helo_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x25f03bc))
================================
I have also checked with several webmail accounts where i am bringing them
in through the same system, and they also seem to identify the correct ip
address to check if the:
debug: SPF: checking HELO
debug: SPF: trimmed HELO down to
lines are the ones to look for.
So one issue i have with the SPF situation is how to get it to check the
correct sending ip address if emails are being forwarded to local accounts?
Secondly it appears that even when it has all the information to do the spf
check, it can't find the module. I thought i had installed it, and when i go
to f:\perl\bin and run "ppm install Mail-SPF-Query" it says:
========================
F:\Perl\bin>ppm install Mail-SPF-Query
Version 1.6 of 'Mail-SPF-Query' is already installed.
Remove it, or use 'verify --upgrade Mail-SPF-Query'.
========================
Should the SPF module work on a windows system? It seems to claim it is
installed.
Having ranted on about how to get SPF working, how do i disable SPF if i
don't get it working, so it doesn't do the unnecessary preparation work?
Sorry, there are rather a lot of questions here, and i expect i show how
little i know about how all of this works by asking them.
Thanks for your help,
Ben
