Re: Re: Keeping logs forever

2013-02-21 Thread Anthony Grimes
the segment size to 1GB, then with 10TB of storage that is only 10k files which should be fine. Adjust the OS open FD limit up a bit if needed. File descriptors don't use too much memory so this should not hurt anything. -Jay On Thu, Feb 21, 2013 at 4:00 PM, Anthony Grimes wrote: Our use ca

Keeping logs forever

2013-02-21 Thread Anthony Grimes
Our use case is that we'd like to log data we don't need away and potentially replay it at some point. We don't want to delete old logs. I googled around a bit and I only discovered this particular post: http://mail-archives.apache.org/mod_mbox/incubator-kafka-users/201210.mbox/%3CCAFbh0Q2=eJcDT