People have done 1M logs/sec
I haven't had access to 10G ethernet to test with, so my tests have been limited
to Gig-E wire speed (~380K log/sec with 250 byte messages)
now, a more complex config may slow this down (slower output modules, lots of
filter rules, regex matches, complex output templates, etc)
David Lang
On Tue, 14 May 2013, Mark Olliver wrote:
Thank you all for your replies, so for now it probably looks best for me to
stick with tcp to enable the load balancing, but it would be great it in the
future there was an option to split the load between servers using REPL.
What is the current max throughput expected for rsyslog?
Regards
Mark
--
Mark Olliver
On 14 May 2013 at 11:48:03, Rainer Gerhards ([email protected]) wrote:
On Mon, May 13, 2013 at 9:03 PM, David Lang <[email protected]> wrote:
On Mon, 13 May 2013, Gary Foster wrote:
On May 13, 2013, at 5:12 AM, Philippe Muller <[email protected]>
wrote:
As far I know, rsyslog doesn't do "load balancing".
Actually, it does play very nicely with load balancers such as HAproxy.
$ActionSendTCPRebindInterval <number>
will configure the TCP forwarding to rebind the TCP connection every
<number> messages which makes it work perfectly through a load balancer.
I'm currently doing exactly this with HAproxy and it works a treat when
using straight TCP output.
However, it seems that the omfwd module is the only module that supports
this configuration. I do not believe RELP or any of the other om* modules
support setting a TCP rebind interval.
Bear in mind, do not set that number too low, you don't want it rebinding
too frequently. I currently rebind after every 2000 message which with my
traffic load is somewhere in the ballpark of 2 seconds generally.
I believe that you are correct across the board. It should not be that
difficult a task to get this feature added to RELP, but with Rainer's crash
project he's not going to get a chance to do anything until June.
This should be pretty close to a cut-n-paste of the code from omtcp to
omrelp, count the messages and close/reopen the connection
the more complex the network protocol the more work there is to set it up,
setting this to give the load balancer a chance to redirect the traffic
every second or two is a good timing to aim for.
I am not sure if it is such a good idea to do this for RELP. While of
course it is possible to do (and probably not so much work), the
performance profile is quite different. There is much more overhead in
creating and shutting down these sessions and I wouldn't do it too
frequently. Anyhow, I am touching RELP currently as well, so I could see if
I add support for this (depending on effort required, obviously not a
priority at the moment). But I am still a bit skeptic...
Rainer
David Lang
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
