Yes, we get a lot of flexibility with omprog.
However, changing a syslog's facility/priority isn't possible right?
Please see below.
if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
action(type="omfile"
action.execOnlyOnceEveryInterval="3600"
template="RSYSLOG_FileFormat")
# Change the facility and message severity forever here for the
next action(s). for example to user.alert
}
# Forward all 'user'.crit and severe to syslog collector.
user.crit action(type="omfwd" target="10.11.12.13" protocol="udp"
port="514" template="MsgForwardFormat")
On Mon, Apr 8, 2024 at 6:37 PM David Lang <da...@lang.hm> wrote:
>
> not easily within rsyslog, with an event correlation engine, you have a lot
> more
> capability (which is why I keep pushing you that way :-) )
>
> you can create a template to use when outputting a log, and that log can say
> anything (it can be arbitrary text unrelated to the log you received), but
> that
> gets really ugly really quickly
>
> you would combine it essentially with a if/else type arrangement to only be
> done
> if the prior action didn't take place. I don't know if the onceeveryinterval
> will work with that.
>
> David Lang
>
>
> On Mon, 8 Apr 2024, Prasad Koya wrote:
>
> > Thank you. "action.execOnlyOnceEveryInterval" is very useful.
> >
> > We intend to keep all unfiltered syslogs in a local file, say file1,
> > but log a rate limited message to file2 and forward that message to a
> > syslog collector.
> >
> > Is it possible to change the facility, severity or message for the next
> > actions?
> >
> > Thanks again.
> >
> > On Sun, Apr 7, 2024 at 11:42 PM David Lang <da...@lang.hm> wrote:
> >>
> >> you would need to do the filtering with omprog on the sending machines.
> >>
> >> I would suggest that rather than throwing them away, you generate a log
> >> every
> >> rate-limiting period along the lines of "X number of messages happened in
> >> the
> >> last period"
> >>
> >> This is really a job for an event correlation engine, since rsyslog is
> >> multi-threaded for performance, it can't count on anything internal seeing
> >> every
> >> message, which is why filtering for the messages and sending them to an
> >> external
> >> app (single threaded or a very complex multi-threaded thing that correlates
> >> across it's threads) is the right thing to do.
> >>
> >> David Lang
> >>
> >> On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:
> >>
> >>> Thanks for the replies.
> >>>
> >>> My requirement is that I have a daemon that may generate a burst of
> >>> syslogs, say, every minute (when a certain subsystem is overloaded).
> >>> We do not want to write all these syslogs to /var/log/messages. We
> >>> also do not want to forward so many of those syslogs to a remote
> >>> syslog server. We wanted to rate limit them, maybe 1 per hour or so.
> >>> For now, I have tried 'omprog' module and sending such syslogs to an
> >>> external program.
> >>>
> >>>
> >>> module( load="omprog" )
> >>>
> >>> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory")
> >>> then {
> >>> action(type="omprog"
> >>> binary="/usr/bin/rate_limit_daemon_syslogs.sh"
> >>> template="RSYSLOG_FileFormat")
> >>> stop
> >>> }
> >>>
> >>> /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
> >>> last syslog forwarded or written locally. If this script decides its
> >>> ok to syslog, then it will call 'logger' with differently worded
> >>> message and different facility.
> >>>
> >>> In our case we'd like to limit these logs at the source and not at an
> >>> intermediate syslog collector. I took a quick look at the sampling
> >>> docs and that may not apply to our scenario.
> >>>
> >>> Thanks.
> >>>
> >>> On Wed, Apr 3, 2024 at 12:07 AM Mariusz Kruk via rsyslog
> >>> <rsyslog@lists.adiscon.com> wrote:
> >>>>
> >>>> A piece of feedback against
> >>>> https://www.rsyslog.com/doc/tutorials/random_sampling.html
> >>>>
> >>>> I know it's obvious for some people but for some it might not be - it
> >>>> will collect _about_ 20% of logs provided the overall number of events
> >>>> is big enough. Since it's a random-based mechanism, it's... random.
> >>>>
> >>>> Additionally, I had a similar thing implemented for load-balancing to
> >>>> different "backends" and for some reason it didn't work uniformly (I
> >>>> think I already wrote about this) - one of four backends was never
> >>>> chosen. The thing is I was choosing not based directly on random(4) but
> >>>> on (random(some_bigger_integer) % 4) which I thought would yield more
> >>>> uniform randomness. It appears I was wrong.
> >>>>
> >>>> MK
> >>>>
> >>>>
> >>>> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
> >>>>> This sounds a bit like you are looking for this:
> >>>>>
> >>>>> https://www.rsyslog.com/doc/tutorials/log_sampling.html
> >>>>>
> >>>>> HTH
> >>>>> Rainer
> >>>>>
> >>>>> El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
> >>>>> (<rsyslog@lists.adiscon.com>) escribió:
> >>>>>> Hi
> >>>>>>
> >>>>>>
> >>>>>> module( load="imuxsock" )
> >>>>>> module( load="imklog" )
> >>>>>>
> >>>>>> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory")
> >>>>>> then {
> >>>>>> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> >>>>>> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> If we have a flurry of, for example, "Out of memory" messages from the
> >>>>>> facility 'kern', we'd like to keep only, say, one per hour in a
> >>>>>> separate file.
> >>>>>>
> >>>>>> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> >>>>>> says do not set low value for queue.size.
> >>>>>>
> >>>>>> Above page also says queue.dequeueslowdown can be used as
> >>>>>> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> >>>>>> above rsyslog.conf working. high and low watermarks are not applicable
> >>>>>> here as it's not a disk based queue. Appreciate any pointers.
> >>>>>>
> >>>>>> Thank you.
> >>>>>> _______________________________________________
> >>>>>> rsyslog mailing list
> >>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>> http://www.rsyslog.com/professional-services/
> >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> >>>>>> if you DON'T LIKE THAT.
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com/professional-services/
> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> >>>>> if you DON'T LIKE THAT.
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com/professional-services/
> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >>>> DON'T LIKE THAT.
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com/professional-services/
> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >>> DON'T LIKE THAT.
> >
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
