There is no guarantee either in the syslog itself that the delivery of
the events will be in the order they were sent. Especially if you're
using UDP.
Also - I don't think there is any guarantee for two queues to be
processed "in parallel".
Regardless of whether you have rsyslog or anything else in the middle,
if you need this level of precision, you need to have sufficiently
precise timestamp so you can order your events by timestamps. Otherwise
it's just an educated guess. That's why TCP has sequence numbers - you
can't rely on the order by which you received packets from the network.
That's the general comment.
In a very particular case - if you're receiving the events from a single
source over a single TCP connection, you can have a reasonable
expectations for rsyslog to receive them in order but I'm not 100% sure
if/how you can make rsyslog make sure it doesn't do any "interleaving"
in terms of output actions. Someone knowing the internals better would
need to elaborate on this more.
You could try pushing all events to a single ruleset with a queue with
just one worker thread. I suppose then the events dequeued from the main
queue and enqueued into the ruleset queue would be in order. I'm not
fully sure about the order of events enqueued into action queues though.
I'm curious myself.
MK
On 13.02.2024 22:43, Prasad Koya via rsyslog wrote:
Hi
Before I explain what I'm trying to solve, here is our rsyslog.conf:
module( load="imuxsock" )
module( load="imklog" )
module( load="impstats" interval="60" severity="7" log.syslog="off"
log.file="/var/log/rsyslog_stats")
$MainMsgQueueTimeoutEnqueue 0
template( name="Msg_ForwardFormat_info" type="list" ) {
constant( value="<166>" )
property( name="timestamp" dateFormat="rfc3164" )
constant( value=" al210 " )
property( name="syslogtag" position.from="1" position.to="32" )
property( name="msg" spifno1stsp="on" )
property( name="msg" )
}
template( name="Msg_ForwardFormat_notice" type="list" ) {
constant( value="<166>" )
property( name="timestamp" dateFormat="rfc3164" )
constant( value=" al210 " )
property( name="syslogtag" position.from="1" position.to="32" )
property( name="msg" spifno1stsp="on" )
property( name="msg" )
}
# Forward to syslog server 1
local4.=info action(type="omfwd" target="10.240.127.159"
protocol="tcp" queue.type="LinkedList" port="34567"
template="Msg_ForwardFormat_info")
local4.=notice action(type="omfwd" target="10.240.127.159"
protocol="tcp" queue.type="LinkedList" port="34567"
template="Msg_ForwardFormat_notice")
# Forward to syslog server 2
local4.=info action(type="omfwd" target="10.240.127.199"
protocol="tcp" queue.type="LinkedList" port="34567"
template="Msg_ForwardFormat_info")
local4.=notice action(type="omfwd" target="10.240.127.199"
protocol="tcp" queue.type="LinkedList" port="34567"
template="Msg_ForwardFormat_notice")
Except for the prefix, the templates Msg_ForwardFormat_Info,
Msg_ForwardFormat_notice are exactly the same. We defined them separately
so we can prefix the message being forwarded with the original
facility/severity tag.
We'd like to use queues in front of the actions so if one TCP connection is
stalled, we do not want forwarding to another TCP connection or logging to
a local file being stalled.
If rsyslog receives 2 messages from 2 tasks in the system, where one is "
local4.info" and the other is "local4.notice", that would result in 4 TCP
connections (due to 4 different action queues/threads above).
Consider this case where an application sends 2 syslogs:
at time t1, a local4.info message
at time t1+few microseconds, a local4.notice message
While these messages are deep copied and enqueued to respective action
queues, it's possible that the second message may reach the remote syslog
server before the first message. I'd like to avoid that situation.
How do I define one queue for all syslogs forwarded to a server
by preserving the facility/severity in each message.
Thank you.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
