In our experience (in older version of the gzip module, which we heavily heavily use), you'll hit threading model issues before you run into gzip compression overheads. We use it for massively large log volumes - which I could put money on blow away yours. ;)
That said, it does take some tuning to get the right performance. Make sure you set your timeout and block size targets appropriate for the volume of content you're compressing. We run with 256k compression blocks and a 10 second timeout. On Fri, Jun 21, 2013 at 10:57 AM, [email protected] <[email protected]> wrote: > On Fri, Jun 21, 2013 at 07:08:47AM -0500, Boylan, James wrote: > > Has anyone made use of the automatic gzip compression in 7.4.1 on a > central logging server? If so, what are your observations regarding > performance? And have you used it in a situation where you have a large > amount of syslog traffic being received by the server? > > > > I'm curious as I currently have a follow up program that compresses > anything older than 4 hours, but as you can imagine it has an overall > impact on the load for the server. I suspect that trying to implement it > through the omfile gzip function would merely offset that load into the > Rsyslog application, but I wanted to see if anyone had direct experience > with it. > > > > Thanks! > > > > --James > > > > Hi James, > > I have not tried it. My one thought is that gzip compression is CPU > intensive and > slow in terms of MB/sec processing speed. That seems like it would not be > particularly > useful on a server processing a LOT of logging. For more moderate use, it > could be > advantageous. Now if gzip were replaced with a higher performing algorithm > like > lz4 or snappy, it could find a use at the top end. > > Regards, > Ken > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
