We are in the planning stages of setting up a rsyslog server pool to accommodate syslog streams from a couple thousand *nix servers; including auditd type data and potentially some application logs (so it's going to be a VERY high volume of data) and we're looking to archive this data somewhere.We have a 10Gb network infrastructure, and I can throw as many RHEL machines at it as needed (as well as F5 load balancers in front).
Eventually the data may need to be searched, but highest priority is getting it written somewhere quickly (and reliably - we need to minimize any possible data loss so our archives can stand up to auditing requirements). In that regard, any suggestions on file systems that can handle that kind of load? Ideally we want all the log files written to the same storage somewhere - i.e. we don't want to have to consolidate files from separate locations to search all the log files for some specific host. On the other hand we can split up load by subnet sources perhaps and route specific machines to specific rsyslog clusters to ease the load on any one cluster (though our larger subnets still may have around 1,000 systems reporting); as long as it's easy to identify where to look for data from a given host. I welcome any advice on setups that allow multiple concurrent (active) rsyslog servers writing to a common-ish file system as well as any gotchas or performance benchmarks we can use to help plan the system. Thanks, Chris Bartram "The purpose of life is not to be happy. It is to be useful, to be honorable, to be compassionate, to have it make some difference that you have lived and lived well". (Ralph Waldo Emerson) _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
