On Thu, 20 Jun 2013, Roberto Giordani wrote:
Hi David,
I'm using TCP and I have 10 threads for MainMsqQueue that receive from 500 to
1000 files for day
I'm using
template(name="filename" type="string"
stringB="/rsyslog-data/%$!path%/%$!server%/%syslogtag%.log")
local6.debug action(type="omfile" DirOwner="%$!owner%" DirCreateMode="0750"
FileCreateMode="0444" DynaFile="filename" )
As you previous suggest in other post "Tuning Rsyslog Centralized...."
Now the performance are excellent and I can decrease numbers of virtual CPU
to be used on Server.
If I reduce threads from
MainMsgQueue from 10 to 5
and use
$DynaFileCacheSize 500
can these options en-queue all msg in the right order.
As long as you have more than one thread it is possible for the messages to be
re-ordered.
Is there some way to en-queue msgs received from a specific listner to a
specific Queue and other msgs received from other listner en-queue to other
Queue?
Yes, you would have to create a ruleset for each listener.
Because I have some servers that sent 1000 msg/second and other 1000
msg/minute, so if the output file will be written first by some thread and
then from the other, queue msg that depends on different queue can avoid
mismatch order in the output file.
If you have two threads writing to the same file you cannot guarantee the order
of the messages.
The problem isn't happening as the messages arrive. Messages from a host get
added to the queue in the order they arrive.
The problem is that when the worker threads pull messages off of the queue they
do so by pulling a batch of messages and then working through the messages.
So you have
thread 1 thread 2
pull messages 1-10 wait
process 1 pull messages 11-20
process 2 process 11
process 3 process 12
process 4 process 13
etc
If you disable threads you will have trouble handling the load because it will
just take too long to process the messages
If you reduce the batch size to 1 you will have problems because the overhead of
locking and unlocking for every message will kill you.
David Lang
I hope to be clear :-)
Regards,
Roberto.
On 06/20/2013 04:00 PM, David Lang wrote:
On Thu, 20 Jun 2013, Roberto Giordani wrote:
Hello,
I've found in some files created on Rsyslog centralized server that
doesn't match timestamp order on source message on client rsyslog.
Is there some correlation with
$MainMsgQueueDequeueBatchSize ?
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:07,189 msg5
...
...
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:06,982 msg1
...
...
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:06,985 msg4
Jun 20 13:35:08 client1 testtag 2013-06-20 13:35:07,048 msg6
Some idea?
There are a number of things that can cause logs to be received in a
different order than they were sent.
If you are using UDP forwarding, the UDP packets may arrive out of order
(especially on dynamicly routed networks). This is true with any version
of syslog
if you relay logs through a box in the middle and have that box highly
available (especially if you have it load balanced), it's possible for
logs to be reordered (this happens with TCP as well as UDP). This is
also true with any version of syslog.
In addition, the fact that rsyslog can have multiple threads working on
processing messages can mean that messages can get processed in a
different order than they were generated. You can avoid this by
configuring rsyslog to limit itself to a single thread, but this
significantly hurts your performance.
Since there are many ways for the logs to be re-ordered, the bottom line
is that your analysis tools need to be able to handle the situation.
It's more common with rsyslog due to the effects of threading, but even
without that it will happen.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
