Log ASN data via default syslog lines

2025-04-04 Thread Georg Faerber
Hi all, I'm using SpamAssassin 4.0.1 and the ASN plugin, which works as expected: relying on add_header all ASN _ASN_ adds a header like X-Spam-ASN: AS200069 Mailjet SAS Besides, I would like to log such results, ideally via the default syslog lines, i.e. no additional "asn"

Re: Log ASN data via default syslog lines

2025-04-02 Thread Tom Hendrikx
Hi, Depending on how you deliver mail after SA has added headers, you might be able to use postfix header checks to log your header,using action INFO or WARN. Something along the likes of: main.cf: header_checks = pcre:/etc/postfix/header_checks header_checks: /^X-Spam-ASN:/ WARN See

Re: double backslash in the log messages

2024-05-22 Thread Vincent Lefevre
02:00 joooj spamd[219339]: config: failed to > > parse line in /srv/d_joooj/home/vinc17/.spamassassin/user_prefs (line > > 192): header LOCAL_TO_LORIA ToCc =~ /loria\\.fr/i > > > > while I just had /loria\.fr/i (with a single backslash) in my > > user_prefs config fi

Re: double backslash in the log messages

2024-05-21 Thread Bill Cole
LORIA ToCc =~ /loria\\.fr/i while I just had /loria\.fr/i (with a single backslash) in my user_prefs config file. Is there a reason to have a double backslash in the log messages or is this a bug? It is intentional to assure that log messages (which may include strings from tainted sources)

double backslash in the log messages

2024-05-21 Thread Vincent Lefevre
Is there a reason to have a double backslash in the log messages or is this a bug? -- Vincent Lefèvre - Web: <https://www.vinc17.net/> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: SA4 spamhaus/DQS "async: aborting" log messages?

2022-12-20 Thread PGNet Dev
Probably asking the obvious, but did you actually substitute "your_dqs_key" with your *actual* DQS key, right? Heh. Not initially -- but yes, since. and, after waiting awhile, i see it's not unique to DQS queries, Tue Dec 20 14:34:27 2022 [916] info: async: aborting after 6.441 s, deadline

Re: SA4 spamhaus/DQS "async: aborting" log messages?

2022-12-20 Thread Riccardo Alfieri
Probably asking the obvious, but did you actually substitute "your_dqs_key" with your *actual* DQS key, right? On 20/12/22 17:26, PGNet Dev wrote: Tue Dec 20 11:16:28 2022 [54384] info: async: aborting after 13.670 s, deadline shrunk: HASHBL, A/compiling.spamassassin.taint.org.your_dqs_key.d

SA4 spamhaus/DQS "async: aborting" log messages?

2022-12-20 Thread PGNet Dev
i've not found anything yet re: what to do about it. with spamd ... --debug=timing,async a detailed log example, ... Tue Dec 20 11:16:14 2022 [54384] dbg: async: starting: SH_ZRD_HEADERS_VERY_FRESH, HASHBL, A/compiling.spamassassin.taint.org.your_dqs_key.zrd.dq.spamhaus.net (time

Re: Spamassassin spamming in log

2022-06-02 Thread Timo Brandt
Thanks, Bert :-) Am 2022-06-02 20:49, schrieb Bert Van de Poel: If you are using systemd, you can "systemctl disable spamd". Otherwise you can indeed use the enabled=0. I would probably do both just in case ;) On 2/06/2022 20:36, Timo Brandt wrote: Maybe one of you has a hint for me how t

Re: Spamassassin spamming in log

2022-06-02 Thread Bert Van de Poel
If you are using systemd, you can "systemctl disable spamd". Otherwise you can indeed use the enabled=0. I would probably do both just in case ;) On 2/06/2022 20:36, Timo Brandt wrote: Maybe one of you has a hint for me how to disable the automatic startup of spamd? Its been a long time ag

Re: Spamassassin spamming in log

2022-06-02 Thread Timo Brandt
Maybe one of you has a hint for me how to disable the automatic startup of spamd? Its been a long time ago that I setup a Debian from scratch :-( It seems that spamd doesnt need to start at system boot so I will disable it. Will this be done when I add ENABLED=0 into the file /etc/defaul

Re: Spamassassin spamming in log

2022-06-02 Thread Timo Brandt
Hi all, indeed - sorry. I wasnt aware of that I do not need to run spamd beside amavis 🥴 Thanks for all your help. Timo Am 2022-06-02 20:18, schrieb Matija Nalis: On Thu, Jun 02, 2022 at 02:47:28PM +0200, Bert Van de Poel wrote: For the errors about nonexistent uses you will want to hav

Re: Spamassassin spamming in log

2022-06-02 Thread Matija Nalis
On Thu, Jun 02, 2022 at 02:47:28PM +0200, Bert Van de Poel wrote: > For the errors about nonexistent uses you will want to have a look at > /etc/default/spamassassin I'm guessing. > For the info messages: this has just got to do with your logging level. You > will want to decrease it in local.cf or

Re: Spamassassin spamming in log

2022-06-02 Thread Matus UHLAR - fantomas
On 02.06.22 14:33, Timo Brandt wrote: I have a running debian 11 with postfix/dovecot and Amavis with clamav / spamassassin. I saw that the spamassassin logfile is growing very fast and found the following entries occuring many times per second. Can you maybe help me to get this fixed? I search

Re: Spamassassin spamming in log

2022-06-02 Thread Benny Pedersen
On 2022-06-02 15:17, Timo Brandt wrote: Hi Bert, I checked the user table: debian-spamd:x:114:120::/var/lib/spamassassin:/usr/sbin/nologin And also adjusted the config file: OPTIONS="-u debian-spamd --create-prefs --max-children 5 --helper-home-dir -s /var/log/spamassassin/spamd.log

Re: Spamassassin spamming in log

2022-06-02 Thread Benny Pedersen
On 2022-06-02 15:13, Bert Van de Poel wrote: For the error: does the spamd user actually exist? that's a requirement of course. to check in shell id spamd I've always controlled SA loglevels through amavis, but from the spamd man page I would expect that it's related to -D. I'm not completel

Re: Spamassassin spamming in log

2022-06-02 Thread Bert Van de Poel
--helper-home-dir -s /var/log/spamassassin/spamd.log But process is already running under root: Am 2022-06-02 15:13, schrieb Bert Van de Poel: For the error: does the spamd user actually exist? that's a requirement of course. I've always controlled SA loglevels through amavis,

Re: Spamassassin spamming in log

2022-06-02 Thread Timo Brandt
Hi Bert, I checked the user table: debian-spamd:x:114:120::/var/lib/spamassassin:/usr/sbin/nologin And also adjusted the config file: OPTIONS="-u debian-spamd --create-prefs --max-children 5 --helper-home-dir -s /var/log/spamassassin/spamd.log But process is already running under

Re: Spamassassin spamming in log

2022-06-02 Thread Bert Van de Poel
an 5, # unless you know what you're doing. OPTIONS="--create-prefs --max-children 5 --helper-home-dir --username spamd --helper-home-dir /home/spamd -s /var/log/spamassassin/spamd.log # Pid file # Where should spamd write its PID to file? If you use the -u or # --username option

Re: Spamassassin spamming in log

2022-06-02 Thread Timo Brandt
be careful! You need to # make sure --max-children is not set to anything higher than 5, # unless you know what you're doing. OPTIONS="--create-prefs --max-children 5 --helper-home-dir --username spamd --helper-home-dir /home/spamd -s /var/log/spamassassin/spamd.log # Pid file # Where sh

Re: Spamassassin spamming in log

2022-06-02 Thread Bert Van de Poel
For the errors about nonexistent uses you will want to have a look at /etc/default/spamassassin I'm guessing. For the info messages: this has just got to do with your logging level. You will want to decrease it in local.cf or maybe also in the default file. On 2/06/2022 14:33, Timo Brandt wrote

Spamassassin spamming in log

2022-06-02 Thread Timo Brandt
Hi all, I have a running debian 11 with postfix/dovecot and Amavis with clamav / spamassassin. I saw that the spamassassin logfile is growing very fast and found the following entries occuring many times per second. Can you maybe help me to get this fixed? I searched along the internet but did

Re: Log reporting spamd[11912]: dns: [...] messages

2022-05-30 Thread Bill Cole
does a substantial number of DNS lookups to check the domains used in email addresses and URLs in the message. The number of these is dependent on your local configuration, but there are many enabled by default. Is there a way to reduce all of these log-lines? (many times longer than the actual

Re: Log reporting spamd[11912]: dns: [...] messages

2022-05-30 Thread DL Neil
dual messages for scanning. > > SpamAssassin does a substantial number of DNS lookups to check the > domains used in email addresses and URLs in the message. The number of > these is dependent on your local configuration, but there are many > enabled by default. > >> I

Re: Log reporting spamd[11912]: dns: [...] messages

2022-05-29 Thread Bill Cole
es a substantial number of DNS lookups to check the domains used in email addresses and URLs in the message. The number of these is dependent on your local configuration, but there are many enabled by default. Is there a way to reduce all of these log-lines? (many times longer than the actual

Re: Log reporting spamd[11912]: dns: [...] messages

2022-05-29 Thread Benny Pedersen
On 2022-05-29 01:25, DL Neil wrote: SpamAssassin x86_64 3.4.0 CentOS 6.el7 release Postfix 2.10.1 unbound 1.6.6 upgrade centos and check you dont added -D or --debug into spamd options more help show spamd options

Re: Log reporting spamd[11912]: dns: [...] messages

2022-05-29 Thread David Bürgin
DL Neil: > SpamAssassin x86_64 3.4.0 CentOS 6.el7 release > Postfix 2.10.1 > unbound 1.6.6 This does not answer your question, but I noticed that the versions you gave are all ~5–8 years old. Often enough such problems disappear after upgrading to a current version.

Log reporting spamd[11912]: dns: [...] messages

2022-05-28 Thread DL Neil
. Have I accidentally released a hydra of services/checks? Is there a way to reduce all of these log-lines? (many times longer than the actual email message itself) Apologies for first-post, learner, ignorance. Web-searching has not revealed the secret. Will appreciate pointers to relevant docs

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread John Hardin
On Fri, 20 Apr 2018, Kevin A. McGrail wrote: If RH/CentOS chose to simply remove those plugins, I would follow like and kind for building the package. +1 -- Kevin A. McGrail Asst. Treasurer & VP Fundraising, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.li

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Kevin A. McGrail
If RH/CentOS chose to simply remove those plugins, I would follow like and kind for building the package. -- Kevin A. McGrail Asst. Treasurer & VP Fundraising, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171 On Fri, Apr 20,

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Reio Remma
Neither spamassassin-3.4.0-2.el7.src.rpm (CentOS 7.4) nor spamassassin-3.4.1-17.fc27.src.rpm have the mentioned files in their source at all. Reio On 20.04.18 17:06, Kevin A. McGrail wrote: Giovanni, I was considering killing it as well. And I was going to look at how CentOS handled this in t

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Kevin A. McGrail
Giovanni, I was considering killing it as well. And I was going to look at how CentOS handled this in the 3.4.1 for their rpms. -- Kevin A. McGrail Asst. Treasurer & VP Fundraising, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.79

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Giovanni Bechis
On 04/20/18 13:53, Kevin A. McGrail wrote: > FYI, I'm well aware of the 3.4 test issue with rulesrc.  I have it symlinked > to a checkout for my purposes.  I'll document that more. > > I am using CentOS 7 as well for testing and not aware of these perl > dependency issues you are having.  Please

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Reio Remma
I suspect rpmbuild gleans the requirements from script files when building. Mail-SpamAssassin-3.4.2/lib/Mail/SpamAssassin/Plugin/RabinKarpBody.pm:use RabinKarpAccel; Mail-SpamAssassin-3.4.2/lib/Mail/SpamAssassin/Util/MemoryDump.pm:use Devel::Size qw(size total_size); Mail-SpamAssassin-3.4.2/mas

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Kevin A. McGrail
FYI, I'm well aware of the 3.4 test issue with rulesrc. I have it symlinked to a checkout for my purposes. I'll document that more. I am using CentOS 7 as well for testing and not aware of these perl dependency issues you are having. Please elaborate further. -- Kevin A. McGrail Asst. Treasure

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Reio Remma
On 20.04.18 9:50, Giovanni Bechis wrote: On 04/19/18 09:24, Reio Remma wrote: [...] *Update:* none of the --option= switches work. handle_user (userdir) unable to find user: '' is caused because I have the -username switch as --username=amavis instead of --username amavis It worked in 3.4.1.

Re: spamc --reporttype= not working and curious log message.

2018-04-20 Thread Reio Remma
On 20.04.18 9:50, Giovanni Bechis wrote: On 04/19/18 09:24, Reio Remma wrote: [...] *Update:* none of the --option= switches work. handle_user (userdir) unable to find user: '' is caused because I have the -username switch as --username=amavis instead of --username amavis It worked in 3.4.1.

Re: spamc --reporttype= not working and curious log message.

2018-04-19 Thread Giovanni Bechis
On 04/19/18 09:24, Reio Remma wrote: [...] > *Update:* none of the --option= switches work. > > handle_user (userdir) unable to find user: '' is caused because I have the > -username switch as --username=amavis instead of --username amavis > > It worked in 3.4.1. > > Is it at all possible that

Re: spamc --reporttype= not working and curious log message.

2018-04-19 Thread Giovanni Bechis
:24 AM, Reio Remma > <mailto:r...@mrstuudio.ee>> wrote: >> >> On 19.04.18 9:45, Reio Remma wrote: >>> Hello! >>> >>> I'm trying to use this to report spam: >>> >>> spamc --reporttype=report --username=amavi

Re: spamc --reporttype= not working and curious log message.

2018-04-19 Thread Reio Remma
pam: spamc --reporttype=report --username=amavis < mail However all I get is: spamc[9632]: Please specify a legal report type It works if I omit the = after --reporttype. This is with SA 3.4.2 from SVN, iirc it worked the other day with --reporttype=report in 3.4.1.

Re: spamc --reporttype= not working and curious log message.

2018-04-19 Thread Kevin A. McGrail
mit the = after --reporttype. This is with SA 3.4.2 from > SVN, iirc it worked the other day with --reporttype=report in 3.4.1. > > I'm also curious about a log message when reporting: > > spamd[9506]: spamd: handle_user (userdir) unable to find user: '' > > > *U

Re: spamc --reporttype= not working and curious log message.

2018-04-19 Thread Reio Remma
iirc it worked the other day with --reporttype=report in 3.4.1. I'm also curious about a log message when reporting: spamd[9506]: spamd: handle_user (userdir) unable to find user: '' *Update:* none of the --option= switches work. handle_user (userdir) unable to find user: 

spamc --reporttype= not working and curious log message.

2018-04-18 Thread Reio Remma
porttype=report in 3.4.1. I'm also curious about a log message when reporting: spamd[9506]: spamd: handle_user (userdir) unable to find user: '' -- Tervitades Reio Remma MR Stuudio 25 aastat *MR Stuudio OÜ* Tondi 17b, 11316, Tallinn Tel +372 650 4808 Mob +372 56 22

Mail log analysis

2017-05-03 Thread David Jones
Does anyone know of a log analysis script that will give summaries of rule hits and average the SA score by sending domain? I am using MailScanner with MailWatch which puts the SA report into a MySQL database along with headers and other email details. This allows me to run some SQL queries

Re: Problem with massive log files

2017-04-05 Thread Matus UHLAR - fantomas
On 05.04.17 11:13, Jim McLachlan wrote: Hi everyone, Thank you all :-) That has sorted out. The spamfilter.sh file was referring to a log file: /var/log/spamassassin/spamd.log The logrotate system was trying to rotate: /var/log/spamd.log I&#x

Re: Problem with massive log files

2017-04-05 Thread Jim McLachlan
Hi everyone, Thank you all :-) That has sorted out. The spamfilter.sh file was referring to a log file: /var/log/spamassassin/spamd.log The logrotate system was trying to rotate: /var/log/spamd.log I've updated spamfilter.sh to point to the

Re: Problem with massive log files

2017-04-05 Thread Jim McLachlan
erver pid: 2756 Sat Oct 15 13:01:32 2016 [2756] info: spamd: server successfully spawned child process, pid 2758 Which matches the lines I caught in the original log snippet. So it is re-dumping a load of old messages back into the mail.log file, but now I know where all that data

Re: Problem with massive log files

2017-04-05 Thread Tom Hendrikx
guess is that spamfilter.sh is writing away log lines to a temporary file for each delivery, and them spewing them again when handling is complete. But there is a bug where spamfilter.sh does not cleanup after itself, and new lines are appended to the existing temporary file, and then the complete

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 9:34 PM, John Hardin wrote: "grep -v" of what? The logged info: lines (assuming they aren't being discarded at the moment)? That does work for identifying hosts, but it won't tell you what's on the other end of the connection. I was just looking for other hosts. I didn't realize

Re: Problem with massive log files

2017-04-04 Thread John Hardin
t it takes a snapshot where tcpdump captures and reports traffic as long as it's running. So a grep -v should give the same info which from a spotcheck of the log snippet isn't going to identify another host. "grep -v" of what? The logged info: lines (assuming they aren&

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
t Oct 15 16:24:54 2016 [2758] info: spamd: connection from ip6-localhost [::1]:56238 to port 783, fd 5 So a grep -v should give the same info which from a spotcheck of the log snippet isn't going to identify another host. regards, KAM

Re: Problem with massive log files

2017-04-04 Thread Martin Gregorie
It occurs to me that anything grinding through enough mail to generate that much logging should also be eating a lot of CPU - so much so that it might even be identified by seeing what is using unexpectedly large amounts of CPU time.  Running 'top' and watching it for a while to see what patterns

Re: Problem with massive log files

2017-04-04 Thread John Hardin
On Wed, 5 Apr 2017, Jim McLachlan wrote: Hi John, That sounds like a good move. I don't have a lot of experience using tcpdump. Could you help prevent me from fumbling around like a wit with it and let me know what I need to do with it to identify the source of the spamd traffic? At

Re: Problem with massive log files

2017-04-04 Thread ap-ml
Usually the directories will exist somewhere in /var or /usr, my linux is rusty, but try this command line in a new terminal window inotifywait -rme modify,attrib,move,close_write,create,delete,delete_self /dname change dname to appropriate directory. inotify is part of iotify-tools on Cento

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 8:51 PM, Jim McLachlan wrote: Thanks. I tried them both with the same results, several e-mail details, then the summary: 61 Kbytes in 8 Requests. They all look like valid e-mails. They are alternatives for the same command. I would expect some entries. 8 sounds about r

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi KAM, Thanks. I tried them both with the same results, several e-mail details, then the summary: 61 Kbytes in 8 Requests. They all look like valid e-mails. Kind regards. Jim. On 05/04/17 01:46, Kevin A. McGrail wrote: On 4/4/2017 8:39 PM, Jim McLachlan wrote:

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
does the logging to maillog return to "normal" levels? I made a copy of spamfilter.sh to my_spamfilter.sh, then did the chmod -x on the original. I updated master.cf to refer to my_spamfilter.sh and restarted postfix and spamassassin. Sadly, no luck. If I tail -f /var/l

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi John, That sounds like a good move. I don't have a lot of experience using tcpdump. Could you help prevent me from fumbling around like a wit with it and let me know what I need to do with it to identify the source of the spamd traffic? Thanks. Kind regards.

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 8:39 PM, Jim McLachlan wrote: Could you let me know where I should look for the temporary files you mentioned? One thing might be postfix queues but I'd expect postfix lines in the maillogs... mailq or postqueue -p Regards, KAM

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
refer to my_spamfilter.sh and restarted postfix and spamassassin. Sadly, no luck. If I tail -f /var/log/mail.log and then send an e-mail, it instantly starts spitting out those log lines. It's also 1:44am here at the moment, so I'm going to have to go to bed now. P

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi ap-ml, This sounds interesting. Could you let me know where I should look for the temporary files you mentioned? I'm on the edges of my knowledge of e-mail and networking here :-) Kind regards. Jim. On 05/04/17 01:11, ap-ml wrote: Its almost as though there is

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 6:42 PM, Jim McLachlan wrote: https://www.digitalocean.com/community/tutorials/how-to-configure-a-mail-server-using-postfix-dovecot-mysql-and-spamassassin More recently, I found this one: https://www.exratione.com/2016/05/a-mailserver-on-ubuntu-16-04-postfix-dovecot-mysql/

Re: Problem with massive log files

2017-04-04 Thread John Hardin
On Tue, 4 Apr 2017, Kevin A. McGrail wrote: On 4/4/2017 8:04 PM, John Hardin wrote: If all else fails, you may want to visit syslog.conf and tell it to ignore mail.info level messages. Hmm, normally I agree with you, John but I'd strongly recommend against that. He's got something hitting

Re: Problem with massive log files

2017-04-04 Thread ap-ml
Its almost as though there is a build-up of messages that are being continually scanned through, I had a similar issue once where due to incorrect permissions, temp files were not being deleted. Perhaps check temp & working directories for such a logjam of emails. Have you also checked for the

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi John, I did that a couple of days ago after I ran out of disk space. It's helped quite a lot, but only in that it's removed a symptom. -rw-r- 1 syslog adm 457498 Apr 5 00:09 /var/log/syslog -rw-r- 1 syslog adm 652564 Apr 4 06:33 /var/log/syslog.1 -rw-r- 1

Re: Problem with massive log files

2017-04-04 Thread Dave Wreski
Hi, I've posted the spamfilter.sh file to http://pasted.co/7b794ccd I don't see anything in there about verbose logging, but there are two lines in there with a resemblance to your suggestion: logger -f $SALOG -p mail.notice -t spamfilter <<<"Spam filter piping to SpamAssassin: $SPAMA

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 8:04 PM, John Hardin wrote: If all else fails, you may want to visit syslog.conf and tell it to ignore mail.info level messages. Hmm, normally I agree with you, John but I'd strongly recommend against that. He's got something hitting spamd approximately 500x more than is needed

Re: Problem with massive log files

2017-04-04 Thread John Hardin
On Wed, 5 Apr 2017, Jim McLachlan wrote: The text "info: spamd: processing message" appears in that 162,761 times. If all else fails, you may want to visit syslog.conf and tell it to ignore mail.info level messages. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 7:58 PM, Jim McLachlan wrote: I'm not sure which message I'm looking for, but for that same file of 1,000,000 lines, I used this line to cut out all occurrences of "postfix" and count them: So ~300 vs 160K or something bizarre... Is there anything using that spamfilter.sh t

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi KAM, I'm not sure which message I'm looking for, but for that same file of 1,000,000 lines, I used this line to cut out all occurrences of "postfix" and count them: $ grep postfix /tmp/mail_sample.log | cut -d " " -f 6- | cut -d "[" -f 1 | sort | uniq -c 7 postfix/cleanup 3 p

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 7:35 PM, Jim McLachlan wrote: The text "info: spamd: processing message" appears in that 162,761 times. Neat... And how many times do you have a line indicating a new message from postfix in the same period? Firewall off port 783 on the box. It's a longshot but perhaps somet

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
I process a million or so a month and my logs are much MUCH smaller with MORE logging enabled? I'm just wondering if you process a stuff load of email and your logs represent that mail volume. I didn't see anything under the "too much logging" category in the log sample.

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
many emails are you processing because I process a million or so a month and my logs are much MUCH smaller with MORE logging enabled? I'm just wondering if you process a stuff load of email and your logs represent that mail volume. I didn't see anything under the "too much loggi

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
p://pasted.co/7b794ccd I don't see anything in there about verbose logging Quick points: 1 - The verbose logging (which I don't think is the issue) would be in your postfix master.cf indicated by -v on smtpd. Reviewing the log snippet, I saw nothing that looked like too much loggi

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 6:53 PM, Jim McLachlan wrote: Do you know why the spamfilter entries in the log file have dates going back to October? Is the normal spamassassin behaviour that isn't usually logged, or is it doing something unusual? It seems to check all of them and log each check

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
le to http://pasted.co/7b794ccd I don't see anything in there about verbose logging Quick points: 1 - The verbose logging (which I don't think is the issue) would be in your postfix master.cf indicated by -v on smtpd. Reviewing the log snippet, I saw nothing that looked like t

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi KAM, No, there's nothing in the master.cf them indicates anything to do with logging verbosely. No occurrences of "-v" and no mention of "log" or logging, etc. Do you know why the spamfilter entries in the log file have dates going back to October? Is the norm

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
ing shows this: logger <<<"Spam filter piping to SpamAssassin, then to: $SENDMAIL $@" ${SPAMASSASSIN} | ${SENDMAIL} "$@" You should be able to comment that out and instead use the following in place: ${SENDMAIL} "$@" Current log file is up to 165 Gb. You s

Re: Problem with massive log files

2017-04-04 Thread Kevin A. McGrail
On 4/4/2017 6:08 PM, Jim McLachlan wrote: I thought spamfilter was spamassassin. No, it's not. It's what we would call the glue. It's a content filter script that is reaching out to a spamassassin daemon called spamd using a lightweight c program called spamc. SpamD allows for spamassas

Re: Problem with massive log files

2017-04-04 Thread Dave Wreski
le it. In fact, a little googling shows this: logger <<<"Spam filter piping to SpamAssassin, then to: $SENDMAIL $@" ${SPAMASSASSIN} | ${SENDMAIL} "$@" You should be able to comment that out and instead use the following in place: ${SENDMAIL} "$@" Curr

Re: Problem with massive log files

2017-04-04 Thread Jim McLachlan
, but not discarded. # # Modified by Jeremy Morton I can post the whole file if it would help. It's only 54 lines. Hope that helps. Current log file is up to 165 Gb. Kind regards. Jim. On 04/04/17 22:41, Dave Wreski wrote: Hi, My s

Re: Problem with massive log files

2017-04-04 Thread Dave Wreski
Hi, My set up consists of Postfix, Postgrey, Spamassassin, Clam-AV, Amavis-new and Dovecot. What is "spamfilter"? Apr 2 10:31:26 oss2 spamfilter: Sun Oct 16 07:24:13 2016 [16208] info: spamd: connection from ip6-localhost [::1]:53930 to port 783, fd 5 What operating system? Regards,

Problem with massive log files

2017-04-04 Thread Jim McLachlan
Hi, I have a problem with the huge amount of messages being logged by spamassassin. I have around 10 active e-mail users on the system, none of whom have any unusual e-mail usage. This is what I've seen in the last 2 hours: $ date Mon 3 Apr 08:00:50 UTC 2017 $ ls -l /var/log/mai

Re: seek-phrases-in-log - does it work correctly?

2017-03-10 Thread Axb
gi?id=6640 isn't properly fixed? ooops something went very belly up with that one. I'll replace and close that bug Thanks for catching this one. Axb Please test latest version of seek-phrases-in-log Works. Thanks for help and patience, I was sure that you will give up with "

Re: seek-phrases-in-log - does it work correctly?

2017-03-10 Thread mar...@mejor.pl
M_MISSP,MSOE_MID_WRONG_CASE,NSL_RCVD_HELO_USER,TO_NO_BRKTS_FROM_MSSP,T_AXB_XM2600,T_BIG_HEADERS_5K,T_CM_XRCVD_VOOZER4,T_FSL_FREEMAIL_1,T_FSL_HELO_NON_FQDN_2,T_HK_MUCHMONEY,T_LOTTO_AGENT,T_SINGLE_HEADER_1K,T_TO_NO_BRKTS_MSFT,__419_FROM_SIG,__ADVANCE_FEE_2_NEW,__ADVANCE_FEE_2_NEW_MONEY,__ADVANCE_FEE_3_NEW

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread Axb
spam. the routine is supposed to create rules based from msgs in your spam folder and needs the ham folder to counterweight against potential FPs so for example, you don't start producing rules based on phrases in disclaimers. in the log, each line starts with Y/N and a score - not sure how necessa

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread Axb
ules based from msgs in your spam folder and needs the ham folder to counterweight against potential FPs so for example, you don't start producing rules based on phrases in disclaimers. in the log, each line starts with Y/N and a score - not sure how necessary it is, I've always had it that wa

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread mar...@mejor.pl
SPACED,__FROM_MISSP_REPLYTO,__FROM_MISSP_URI,__FROM_RUNON,__FSL_419_1,__FSL_419_2,__FSL_419_3,__FSL_419_4,__FSL_419_5,__FSL_HELO_USER_1,__FSL_HELO_USER_3,__FSL_UA_2,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_DATE,__HAS_FROM,__HAS_MESSAGE_ID,__HAS_MIMEOLE,__HAS_MSGID,__HAS_MSMAIL_PRI,__HAS_RCVD,__HAS_REPL

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread Axb
Why do I need to check mails-that-i-classified-as-spam-or-ham against rules? If I understand how creating auto rules works masscheck only dumps strings from ham and spam. the routine is supposed to create rules based from msgs in your spam folder and needs the ham folder to counterweight against p

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread mar...@mejor.pl
,__LOTTO_ADMITS_1,__LOTTO_WIN_01,__MIMEOLE_MS,__MIME_VERSION,__MISSING_REF,__MISSING_REPLY,__MISSING_THREAD,__MONEY_FRAUD,__MONEY_FRAUD_3,__MONEY_FRAUD_5,__MONEY_LOTTERY,__MSGID_OK_DIGITS,__MSOE_MID_WRONG_CASE,__M_NOTIFIC,__NAKED_TO,__NONEMPTY_BODY,__NO_INR_YES_REF,__OE_MUA,__RCVD_VIA_APNIC_E,__RCVD_VI

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread mar...@mejor.pl
DNS_SHORT,__REPLYTO_EXISTS,__REPLY_FREEMAIL,__SANE_MSGID,__SARE_FRAUD_BARRISTER,__SINGLE_HEADER_1K,__SUBJ_2UPPER,__SUBJ_4LOWER,__SUBJ_HAS_WORDS,__SUBJ_NOT_SHORT,__TOCC_EXISTS,__TO_NO_ARROWS_R,__TO_NO_BRKTS_FROM_MSSP,__TO_NO_BRKTS_FROM_RUNON,__TO_NO_BRKTS_MSFT,__TO_NO_BRKTS_NOTLIST,__TVD_BODY,__TVD_MI

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread Axb
s strings from ham and spam. the routine is supposed to create rules based from msgs in your spam folder and needs the ham folder to counterweight against potential FPs so for example, you don't start producing rules based on phrases in disclaimers. in the log, each line starts with

Re: seek-phrases-in-log - does it work correctly?

2017-03-09 Thread mar...@mejor.pl
_HEADER_1K,__SUBJ_2UPPER,__SUBJ_4LOWER,__SUBJ_HAS_WORDS,__SUBJ_NOT_SHORT,__TOCC_EXISTS,__TO_NO_ARROWS_R,__TO_NO_BRKTS_FROM_MSSP,__TO_NO_BRKTS_FROM_RUNON,__TO_NO_BRKTS_MSFT,__TO_NO_BRKTS_NOTLIST,__TVD_BODY,__TVD_MIME_ATT_TP,__URI_MAILTO,__XM_MSOE6,__XM_MS_IN_GENERAL,__XM_OUTLOOK_EXPRESS,__XPRIO,__YO

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread Axb
On 03/08/2017 04:55 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 16:33, Axb pisze: On 03/08/2017 04:16 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 16:06, Axb pisze: On 03/08/2017 03:58 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 15:27, Axb pisze: As your command below shows you're us

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread mar...@mejor.pl
W dniu 08.03.2017 o 16:33, Axb pisze: > On 03/08/2017 04:16 PM, mar...@mejor.pl wrote: >> W dniu 08.03.2017 o 16:06, Axb pisze: >>> On 03/08/2017 03:58 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 15:27, Axb pisze: > As your command below shows you're using --reqpatlength 0 > > S

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread Axb
On 03/08/2017 04:16 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 16:06, Axb pisze: On 03/08/2017 03:58 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 15:27, Axb pisze: As your command below shows you're using --reqpatlength 0 Start off with some sane as for example --reqpatlength 40 you ma

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread mar...@mejor.pl
W dniu 08.03.2017 o 16:06, Axb pisze: > On 03/08/2017 03:58 PM, mar...@mejor.pl wrote: >> W dniu 08.03.2017 o 15:27, Axb pisze: >>> As your command below shows you're using --reqpatlength 0 >>> >>> Start off with some sane as for example --reqpatlength 40 >>> >>> you may also want to play with --ma

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread Axb
On 03/08/2017 03:58 PM, mar...@mejor.pl wrote: W dniu 08.03.2017 o 15:27, Axb pisze: As your command below shows you're using --reqpatlength 0 Start off with some sane as for example --reqpatlength 40 you may also want to play with --maxtextread ( I use --maxtextread 8192 for FRAUD rules) B

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread mar...@mejor.pl
W dniu 08.03.2017 o 15:27, Axb pisze: > As your command below shows you're using --reqpatlength 0 > > Start off with some sane as for example --reqpatlength 40 > > you may also want to play with --maxtextread > ( I use --maxtextread 8192 for FRAUD rules) But with --reqpatlength 10, 40, 100 or 1

Re: seek-phrases-in-log - does it work correctly?

2017-03-08 Thread Axb
s/rule-dev/seek-phrases-in-log --reqpatlength I'm not sure if it works correctly, please look: $ /home/masscheck/spamassassin-trunk//masses/rule-dev/seek-phrases-in-log --ham /home/masscheck/auto/tmp/all_w.h --spam /home/masscheck/auto/tmp/all_w.s --rules --ruleprefix __SEEK_FRAUD_ --req

  1   2   3   4   >