My solution is
queue.type="LinkedList"
queue.size="4096"
queue.timeoutEnqueue="0" # timeout for reject new messages if queue
is full
queue.dequeuebatchSize="4000"
ср, 7 февр. 2024 г. в 09:03, Prasad Koya via rsyslog <
rsyslog@lists.adiscon.com>:
> Alex, excuse me for using this thread for posting my question. But my
> question is also on the same topic.
>
> I have configured rsyslogd (v8.2102.0) to forward syslogs to two remote
> servers, one using TCP protocol and second one using UDP. Besides
> forwarding, I also have a rule to log to a local file. Here are the
> relevant sections from my rsyslog.conf. I can post the full file if needed.
>
>
>
> $MainMsgQueueTimeoutEnqueue 0
>
> local4.* action(type="omfwd" target="10.240.219.64"
> action.resumeRetryCount="10" protocol="tcp" port="36456"
> template="Syslogs_ForwardFormat")
> local4.* action(type="omfwd" target="10.16.39.124"
> protocol="udp" port="36456" template="Syslogs_ForwardFormat")
>
> *.* /var/log/syslogs
>
> After the TCP connection is established and messages are getting forwarded,
> say someone pulls out the ethernet cable at the remote syslog collector 1
> that is using tcp protocol. Some number of syslogs from this point are
> still forwarded to the server 2 using udp protocol and written to the local
> file as well. However if the TCP connection to the first server does not
> recover then after some time, we do not see any syslogs in the local file
> or at the second syslog server. I straced rsyslogd and all I see is the
> recvmsg() calls on the /dev/log socket.
>
> How do we tell rsyslog to drop the TCP connection if the other end is no
> longer receiving the syslogs. Perhaps it can drop the connection and retry
> after say X minutes. I tried using resumeRetryCount as documented at
> https://www.rsyslog.com/doc/configuration/actions.html, but that is not
> helping in my situation. Perhaps I'm missing some other "action" setting?
> Appreciate if you can give me a pointer to a sample configuration or point
> me to relevant documentation.
>
> Thank you.
>
>
>
>
> On Mon, Feb 5, 2024 at 11:11 AM David Lang via rsyslog <
> rsyslog@lists.adiscon.com> wrote:
>
> > you have a queue of 1024 for rabbitmq, if there are more messages than
> > that
> > pending, other processing will stop until the queue can accept more
> > messages.
> > Setup a larger queue (potentially a disk assisted queue) to handle longer
> > outages.
> >
> > you may also want to consider configuring the queue to throw away
> messages
> > if it
> > gets too full.
> >
> > David Lang
> >
> > On Mon, 5 Feb 2024, Alex via rsyslog wrote:
> >
> > > I have rsyslog (8.2310.0) in docker container from latest alpine image
> > as a
> > > syslog collector with forwarding to logstash.local and rabbitmq.local
> > >
> > > Rsyslog stops working when rabbitmq is unavailable and work fine when
> > > rabbitmq server is up
> > >
> > > How to configure rsyslog to work when the destination is unavailable ?
> > >
> > >
> > > module(load="omrabbitmq")
> > > module(load="imptcp" threads="3")
> > > input(type="imptcp" port="514" ruleset="syslogCollector")
> > >
> > > template(
> > > name="json_syslog"
> > > type="list"
> > > option.json="on"
> > > ) {
> > > constant(value="{")
> > > constant(value="\"@timestamp\":\"")
> > > property(name="timereported" dateFormat="rfc3339")
> > > constant(value="\",\"type\":\"syslog_json")
> > > constant(value="\",\"tag\":\"")
> > > property(name="syslogtag" format="json")
> > > constant(value="\",\"relayhost\":\"") property(name="fromhost")
> > > constant(value="\",\"relayip\":\"")
> property(name="fromhost-ip")
> > > constant(value="\",\"logsource\":\"") property(name="source")
> > > constant(value="\",\"hostname\":\"") property(name="hostname"
> > > caseconversion="lower")
> > > constant(value="\",\"program\":\"")
> property(name="programname")
> > > constant(value="\",\"source\":\"") property(name="app-name"
> > > caseConversion="lower" onEmpty="null")
> > > constant(value="\",\"priority\":\"") property(name="pri")
> > > constant(value="\",\"severity\":\"")
> > > property(name="syslogseverity" caseConversion="upper")
> > > constant(value="\",\"facility\":\"")
> > property(name="syslogfacility")
> > > constant(value="\",\"severity_label\":\"")
> > > property(name="syslogseverity-text")
> > > constant(value="\",\"facility_label\":\"")
> > > property(name="syslogfacility-text")
> > > constant(value="\",\"message\":\"") property(name="msg"
> > format="json")
> > > constant(value="\",\"end_msg\":\"")
> > > constant(value="\"}\n")
> > > }
> > >
> > > ruleset(
> > > name="syslogCollector"
> > > ) {
> > > @@elk.local:5000;json_syslog
> > > call send2mqtt
> > > }
> > >
> > > ruleset (
> > > name="send2mqtt"
> > >
> > > queue.type="LinkedList"
> > > queue.size="1024"
> > > queue.dequeueBatchSize="512"
> > > queue.filename="q_mqtt"
> > > queue.saveonshutdown="off"
> > > queue.highwatermark="450"
> > > queue.lowwatermark="50"
> > >
> > > ) {
> > > action(
> > > type="omrabbitmq"
> > > host="rabbitmq.local"
> > > verify_peer="off"
> > > verify_hostname="off"
> > > virtual_host="/"
> > > user="xxx"
> > > password="xxx"
> > > exchange="syslog"
> > > routing_key="messages"
> > > body_template="json_syslog"
> > >
> > > action.resumeRetryCount="0"
> > > action.reportSuspension="on"
> > > action.reportSuspensionContinuation="on"
> > > action.resumeInterval="10"
> > > )
> > > }
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> > >
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> >
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
