rsyslog processes messages one at a time, so there isn't state from prior
messages to reference (in the general case), and messages may be processed in
parallel.
that being said, rsyslog does have 'global variables' that persist from message
to message that could be used for something like this, but be aware that there
is a performance penalty for using them.
I would suggest that you use a tool better designed for the task, Simple Event
Correlator is my go-to tool for anything where I need to do analysis across
multiple messages. To keep SEC from being a bottleneck, I use rsyslog to filter
and direct the logs so that each instance of SEC only gets messages that it is
going to care about, rather than sending it everything and letting it filter it.
David Lang
On Wed, 6 Jul 2022, Thangalin via rsyslog wrote:
Date: Wed, 6 Jul 2022 14:01:13 -0700
From: Thangalin via rsyslog <rsyslog@lists.adiscon.com>
To: rsyslog@lists.adiscon.com
Cc: Thangalin <thanga...@gmail.com>
Subject: [rsyslog] Suppress duplicate messages
A stateless application starts, runs, and stops every 1 second. If the
application encounters an error, that error is logged. The system is
running rsyslogd 8.1901.0.
When the error occurs, the log is spammed every 1 second until the error is
resolved. We only need to see the error once. The application, being
stateless, does not "remember" that it had already logged the error.
The $RepeatedMsgReduction on option is not a recommended solution, for
various reasons.
Here's a first attempt:
if $msg contains "app error" then set $/app!error!count = $/app!error!count
+ 1;
if $/app!error!count == 10 then set $/app!error!count = 1;
if not $/app!error!count == 1 then ~;
Then:
logger "app error"
logger "app error"
sudo tail -f /var/log/syslog
Shows:
2022-07-06T13:24:42.697247-07:00 host username: app error
2022-07-06T13:24:42.880985-07:00 host username: app error
How would you codify something like the following logic in rsyslog's
scripting language:
IF NOT DEFINED error_count THEN error_count = 0
IF message CONTAINS "{{error message}}" THEN error_count = error_count
+ 1
IF error_count > 10 THEN error_count = 1
IF error_count != 1 THEN stop
Essentially, the first time the error is encountered, log the message;
otherwise, when the error message has occurred 10 times, log the error
again. In effect, this will log the error once every 10 seconds. A more
rigorous solution would also take into consideration the timestamp to reset
the count. Possibly:
IF NOT DEFINED error_count THEN error_count = 0
IF NOT DEFINED error_timestamp THEN error_timestamp = current_timestamp
IF message CONTAINS "{{error message}}" THEN {error_count = error_count
+ 1; error_timestamp = current_timestamp}
IF current_timestamp - error_timestamp > 10 seconds THEN {error_count =
1; error_timestamp = current_timestamp}
IF error_count > 10 THEN error_count = 1
IF error_count != 1 THEN stop
Thank you!
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
