I have rechecked it... as noted before, this "message repeated" string
appears nowhere in the packet capture I took. Also the source is not
repeating any of those messages. It must be rsyslog thinking there are
duplicates and inserting this message.
What would the empty square brackets trailing the message mean?
The interesting thing is that it indeed only happens with the messages
from Checkpoint Smartcenter / Log Exporter. What's special about it is
that it's the only source in our setup sending RFC5424-style messages
containing a structured data field.
The general interesting thing in our setup is that we're using multiple
queues.
Was there any bug of consequence somehow related to all of this in or
after 8.33.1 (which is the latest one available in SLES)?
Thanks.
On 10/11/2019 8:37 PM, David Lang wrote:
repeated message reduction is not recommended. It is a feature that
was useful back when it was only people looking at the logs, but when
you are analyzing the logs with automation, the 'message repeated'
logs reduce your accuracy, not improve it.
that 'message repeated' is happening on the sender, so look to fix it
there (in rsyslog we started having the message repeated log include
the log being repeated so that it was easier to track what's happening)
David Lang
On Fri, 11 Oct 2019, Marki via rsyslog wrote:
Date: Fri, 11 Oct 2019 19:14:27 +0200
From: Marki via rsyslog <[email protected]>
To: [email protected]
Cc: Marki <[email protected]>
Subject: [rsyslog] Rsyslog seems to think there are duplicates
Hey,
We are using Checkpoint Log Exporter
(https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk122323)
I don't think that it matters, except maybe for the fact that they
are implementing RFC5424.
The logs are being written to file by Rsyslog as follows:
---------------------
module(load="imtcp")
module(load="imudp")
input(type="imtcp" port="514")
input(type="imudp" port="514")
template(name="FileNameByHost" type="string"
string="/syslog/%fromhost-ip%.log")
if ($fromhost-ip startswith "192.168.x.y") then { # RFC5424
action(type="omfile" dynaFile="FileNameByHost"
template="RSYSLOG_SyslogProtocol23Format") <-------------------
} else {
action(type="omfile" dynaFile="FileNameByHost"
template="RSYSLOG_FileFormat")
}
call asyncelk
...
---------------------
Now Rsyslog thinks some messages are duplicates. And apparently it
thinks there are _many_ duplicates. I have sniffed the traffic, and
no duplicates are being transmitted AFAICS. Here is an example of
what is logged nevertheless:
<134>1 2019-10-11T16:27:51Z chkpt-mgt CheckPoint 15439 -
[action:"Reject"; ifdir:"inbound"; ifname:"eth2.53"; logid:"0";
loguid:"{0x0,0x0,0x0,0x0}"; origin:"192.168.1.2";
originsicname:"xxxxx"; sequencenum:"32"; time:"1570811271";
version:"5"; dst:"224.0.0.18"; hll_key:"2008789500278145398";
inzone:"Internal"; layer_name:"Unified"; layer_uuid:"a-b-c-d-e";
match_id:"177"; parent_rule:"0"; rule_action:"Reject";
rule_name:"xyz"; rule_uid:"a-b-c-d-e"; product:"VPN-1 & FireWall-1";
proto:"51"; service_id:"AH"; src:"1.2.3.4"; ] message repeated 49
times: []
Other times it says "99 times" or "149 times", ... I've created somes
stats in fact :D
# tail -n 10000 //syslog/1.2.3.4.log | grep -o -E 'repeated [0-9]+
times' | sort | uniq -c | sort -n | tail -10
23 repeated 44 times
26 repeated 199 times
35 repeated 159 times
37 repeated 9 times
38 repeated 4 times
101 repeated 149 times
155 repeated 109 times
408 repeated 59 times
417 repeated 99 times
2926 repeated 49 times
This is SLES 15-SP1 running rsyslog-8.33.1-3.17.1.x86_64
Any idea?
Thanks,
Marki
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
