not easily within rsyslog, with an event correlation engine, you have a lot more
capability (which is why I keep pushing you that way :-) )
you can create a template to use when outputting a log, and that log can say
anything (it can be arbitrary text unrelated to the log you received), but that
gets really ugly really quickly
you would combine it essentially with a if/else type arrangement to only be done
if the prior action didn't take place. I don't know if the onceeveryinterval
will work with that.
David Lang
On Mon, 8 Apr 2024, Prasad Koya wrote:
Thank you. "action.execOnlyOnceEveryInterval" is very useful.
We intend to keep all unfiltered syslogs in a local file, say file1,
but log a rate limited message to file2 and forward that message to a
syslog collector.
Is it possible to change the facility, severity or message for the next actions?
Thanks again.
On Sun, Apr 7, 2024 at 11:42 PM David Lang <da...@lang.hm> wrote:
you would need to do the filtering with omprog on the sending machines.
I would suggest that rather than throwing them away, you generate a log every
rate-limiting period along the lines of "X number of messages happened in the
last period"
This is really a job for an event correlation engine, since rsyslog is
multi-threaded for performance, it can't count on anything internal seeing every
message, which is why filtering for the messages and sending them to an external
app (single threaded or a very complex multi-threaded thing that correlates
across it's threads) is the right thing to do.
David Lang
On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:
Thanks for the replies.
My requirement is that I have a daemon that may generate a burst of
syslogs, say, every minute (when a certain subsystem is overloaded).
We do not want to write all these syslogs to /var/log/messages. We
also do not want to forward so many of those syslogs to a remote
syslog server. We wanted to rate limit them, maybe 1 per hour or so.
For now, I have tried 'omprog' module and sending such syslogs to an
external program.
module( load="omprog" )
if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
action(type="omprog"
binary="/usr/bin/rate_limit_daemon_syslogs.sh"
template="RSYSLOG_FileFormat")
stop
}
/usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
last syslog forwarded or written locally. If this script decides its
ok to syslog, then it will call 'logger' with differently worded
message and different facility.
In our case we'd like to limit these logs at the source and not at an
intermediate syslog collector. I took a quick look at the sampling
docs and that may not apply to our scenario.
Thanks.
On Wed, Apr 3, 2024 at 12:07 AM Mariusz Kruk via rsyslog
<rsyslog@lists.adiscon.com> wrote:
A piece of feedback against
https://www.rsyslog.com/doc/tutorials/random_sampling.html
I know it's obvious for some people but for some it might not be - it
will collect _about_ 20% of logs provided the overall number of events
is big enough. Since it's a random-based mechanism, it's... random.
Additionally, I had a similar thing implemented for load-balancing to
different "backends" and for some reason it didn't work uniformly (I
think I already wrote about this) - one of four backends was never
chosen. The thing is I was choosing not based directly on random(4) but
on (random(some_bigger_integer) % 4) which I thought would yield more
uniform randomness. It appears I was wrong.
MK
On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
This sounds a bit like you are looking for this:
https://www.rsyslog.com/doc/tutorials/log_sampling.html
HTH
Rainer
El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
Hi
module( load="imuxsock" )
module( load="imklog" )
if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
action(type="omfile" queue.size="1000" queue.type="LinkedList"
queue.dequeueSlowDown="3600000000" file="/var/log/oom")
}
If we have a flurry of, for example, "Out of memory" messages from the
facility 'kern', we'd like to keep only, say, one per hour in a
separate file.
https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
says do not set low value for queue.size.
Above page also says queue.dequeueslowdown can be used as
rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
above rsyslog.conf working. high and low watermarks are not applicable
here as it's not a disk based queue. Appreciate any pointers.
Thank you.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
