Hi David,
I'm using TCP and I have 10 threads for MainMsqQueue that receive from
500 to 1000 files for day
I'm using
template(name="filename" type="string"
stringB="/rsyslog-data/%$!path%/%$!server%/%syslogtag%.log")
local6.debug action(type="omfile" DirOwner="%$!owner%"
DirCreateMode="0750" FileCreateMode="0444" DynaFile="filename" )
As you previous suggest in other post "Tuning Rsyslog Centralized...."
Now the performance are excellent and I can decrease numbers of virtual
CPU to be used on Server.
If I reduce threads from
MainMsgQueue from 10 to 5
and use
$DynaFileCacheSize 500
can these options en-queue all msg in the right order.
Is there some way to en-queue msgs received from a specific listner to
a specific Queue and other msgs received from other listner en-queue to
other Queue?
Because I have some servers that sent 1000 msg/second and other 1000
msg/minute, so if the output file will be written first by some thread
and then from the other, queue msg that depends on different queue can
avoid mismatch order in the output file.
I hope to be clear :-)
Regards,
Roberto.
On 06/20/2013 04:00 PM, David Lang wrote:
On Thu, 20 Jun 2013, Roberto Giordani wrote:
Hello,
I've found in some files created on Rsyslog centralized server that
doesn't match timestamp order on source message on client rsyslog.
Is there some correlation with
$MainMsgQueueDequeueBatchSize ?
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:07,189 msg5
...
...
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:06,982 msg1
...
...
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:06,985 msg4
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:07,048 msg6
Some idea?
There are a number of things that can cause logs to be received in a
different order than they were sent.
If you are using UDP forwarding, the UDP packets may arrive out of order
(especially on dynamicly routed networks). This is true with any version
of syslog
if you relay logs through a box in the middle and have that box highly
available (especially if you have it load balanced), it's possible for
logs to be reordered (this happens with TCP as well as UDP). This is
also true with any version of syslog.
In addition, the fact that rsyslog can have multiple threads working on
processing messages can mean that messages can get processed in a
different order than they were generated. You can avoid this by
configuring rsyslog to limit itself to a single thread, but this
significantly hurts your performance.
Since there are many ways for the logs to be re-ordered, the bottom line
is that your analysis tools need to be able to handle the situation.
It's more common with rsyslog due to the effects of threading, but even
without that it will happen.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
