Hello, we have a 3 node kafka cluster setup, with quite a bunch of topics, that have a nice life, regular cleans, compact, etc. 1 topic however keeps growing, new segments are recreated at regular intervals, but old segments are never deleted.
Our setup is based on Confluent 3.2.0 OSS, hence Apache Kafka 0.10.2.0. The server.props related to log retention are fairly straightforward: log.flush.interval.messages=10000 log.flush.interval.ms=1000 log.index.interval.bytes=4096 log.index.size.max.bytes=10485760 log.retention.check.interval.ms=300000 log.retention.bytes=-1 log.retention.ms=7200000 log.segment.bytes=1073741824 We do apply specific configurations for each topic to ensure proper lifetime, cleanup etc. For the topic at hand, those are retention.ms=3600001 segment.ms=3600000 compression.type=uncompressed cleanup.policy=delete I tried quite a bunch of configurations to try various setups, but I'm in a dead end and my final resort would be to delete the topic (and all data underneath) and just recreate. I still can because this is a test/staging environment. I'm a bit reluctant as I'd like to understand what's going on because if this hits us on a production system ... Any help or insights would be much appreciated! -wim
