As a practical matter, if you have multiple senders, you don't need to have
'correct' load balancing where each sender opens multiple connections and sends
to all targets.
Instead you can get by with a much simpler mechanism.
setup a load balancer of your choice, use TCP, and configure the senders to
disconnect and reconnect every X messages (set X to something large enough that
the connection, slow start, and encryption handshake isn't significant, start at
something like 1000, or 10000, a few seconds worth of logs)
with many senders, your load will be fairly well balanced statistically, and by
disconnecting and reconnecting you give the load balancer a chance to adjust the
inbalance.
It won't be perfect, but in practice it's pretty close.
Rsyslog supports this on the sender side (rebind interval) for exactly this
purpose, and you can use anything from CLUSTERIP with corosync/pacemaker to
external load balancers to split the traffic across multiple systems with
failover if a system stops responding.
I've done this with very high volume systems
re: DTLS, to be able to do TLS over UDP, you have to introduce sequencing and
retries, at which point you are pretty much reinventing TCP.
If these connections are over unreliable networks (such as the Internet), I
highly recommend that you look at RELP as a reasonable protocol. That way if the
connection gets broken, you don't lose any logs.
David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
