I have a multi-layered setup with a central rsyslog-based
"load-balancer". Said load balancer reaches up to some 8-10 kEPS without
breaking a sweat (other components which do some heavier filtering and
http output tend to be significantly more stressed). I user RELP with
TLS (mutual authentication). It has 16CPUs but the load rarely goes
above 5 or so.
On 01.07.2022 13:40, Brendan Kearney via rsyslog wrote:
I have both RELP (TCP/20514) and traditional syslog (UDP/514) setup on
two hosts. the RELP instances are load balanced behind a single IP on
HAProxy for high availability and fault tolerance. the syslog
instances are load sharing behind an Anycast IP, so they both listen
on the same IP, but are not in conflict because of the Anycast. in
all cases, the log receivers insert the records into a multi-master
MariaDB database on 3 nodes that is behind HAProxy as well.
i dont see anywhere near 380k messages per minute, but do have about
440k messages per day. You can probably tune and tweak to support
that message rate with more instances and better gear than i have.
Without a database backend, you can have all your log in one place,
without having to work about file system locking and contention.
i think i have seen RELP plus TLS, and maybe RELP plus GSSAPI, which
may offer encryption in transit. you can secure the database access
with TLS, too, and do encyption within the database for encryption at
rest. I am not sure if that is easier or better than file system
encryption via LUKS, etc. I dont have any of that going on.
On 7/1/22 6:59 AM, Dimi Onobodies via rsyslog wrote:
That could also be a solution. Iv seen numerous implementations using
RELP.
Thanks everyone for sharing your thoughts
________________________________
From: rsyslog <rsyslog-boun...@lists.adiscon.com> on behalf of
Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com>
Sent: Thursday, June 30, 2022 1:42 PM
To: rsyslog@lists.adiscon.com <rsyslog@lists.adiscon.com>
Cc: Mariusz Kruk <k...@epsilon.eu.org>
Subject: Re: [rsyslog] rsyslog cluster with NFS backend
I'd simply forward from two remote rsyslogs to a central one using RELP
and not bother with NFS and such. The upside is that you can scale this
easily to other non-local installation (it's way easier to add another
RELP source than to export your NFS to some external site).
On 30.06.2022 12:36, Dimi Onobodies via rsyslog wrote:
Hi.
I have a single rsyslog server (on cloud) receiving logs from a
number of systems (also in cloud). I am planning to add a second
server and use NFS cloud storage as the backend. This storage will
be shared across the two syslog systems.
At the moment, the single rsyslog system receives around 380.000
messages per minute and stores them on local disk.
The reason for opting NFS shared storage is that logs from systems
will be on one location instead of two. For example, without shared
storage and due to load balancing, Monday logs from serverA can be
found in rsyslog1 server and Tuesday logs for same server can be on
rsyslog2 server.
However, I am thinking about the performance of NFS and the
(negative) impact that it could cause on rsyslog. I am concerned about:
* NFS deadlocks
* behaviour when two rsyslog processes write to same file
* encryption and network delay of NFS backend
* rotation of files
Has anyone implemented such rsyslog setup and found it problematic?
Any thoughts/comments for such setup?
Thanks
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
