I have both RELP (TCP/20514) and traditional syslog (UDP/514) setup on
two hosts. the RELP instances are load balanced behind a single IP on
HAProxy for high availability and fault tolerance. the syslog instances
are load sharing behind an Anycast IP, so they both listen on the same
IP, but are not in conflict because of the Anycast. in all cases, the
log receivers insert the records into a multi-master MariaDB database on
3 nodes that is behind HAProxy as well.
i dont see anywhere near 380k messages per minute, but do have about
440k messages per day. You can probably tune and tweak to support that
message rate with more instances and better gear than i have. Without a
database backend, you can have all your log in one place, without having
to work about file system locking and contention.
i think i have seen RELP plus TLS, and maybe RELP plus GSSAPI, which may
offer encryption in transit. you can secure the database access with
TLS, too, and do encyption within the database for encryption at rest.
I am not sure if that is easier or better than file system encryption
via LUKS, etc. I dont have any of that going on.
On 7/1/22 6:59 AM, Dimi Onobodies via rsyslog wrote:
That could also be a solution. Iv seen numerous implementations using RELP.
Thanks everyone for sharing your thoughts
________________________________
From: rsyslog <rsyslog-boun...@lists.adiscon.com> on behalf of Mariusz Kruk via
rsyslog <rsyslog@lists.adiscon.com>
Sent: Thursday, June 30, 2022 1:42 PM
To: rsyslog@lists.adiscon.com <rsyslog@lists.adiscon.com>
Cc: Mariusz Kruk <k...@epsilon.eu.org>
Subject: Re: [rsyslog] rsyslog cluster with NFS backend
I'd simply forward from two remote rsyslogs to a central one using RELP
and not bother with NFS and such. The upside is that you can scale this
easily to other non-local installation (it's way easier to add another
RELP source than to export your NFS to some external site).
On 30.06.2022 12:36, Dimi Onobodies via rsyslog wrote:
Hi.
I have a single rsyslog server (on cloud) receiving logs from a number of
systems (also in cloud). I am planning to add a second server and use NFS cloud
storage as the backend. This storage will be shared across the two syslog
systems.
At the moment, the single rsyslog system receives around 380.000 messages per
minute and stores them on local disk.
The reason for opting NFS shared storage is that logs from systems will be on
one location instead of two. For example, without shared storage and due to
load balancing, Monday logs from serverA can be found in rsyslog1 server and
Tuesday logs for same server can be on rsyslog2 server.
However, I am thinking about the performance of NFS and the (negative) impact
that it could cause on rsyslog. I am concerned about:
* NFS deadlocks
* behaviour when two rsyslog processes write to same file
* encryption and network delay of NFS backend
* rotation of files
Has anyone implemented such rsyslog setup and found it problematic?
Any thoughts/comments for such setup?
Thanks
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
