Hello, Please find the server.properties info. The Kafka server is running on a Ubuntu 14.XX instance.
I have removed all the commented section. So the data I post below would be the only properties we are using for our server. All the other properties are commented(so I guess the default values are being used). Also just to add I can see in the log files that topic index segments are getting deleted at regular intervals. I am only having issues with the consumer offset topic. --------------------------------------------------------------------------- broker.id=0 Switch to enable topic deletion or not, default value is false #delete.topic.enable=true num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/tmp/kafka-logs num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=168 log.segment.bytes=1073741824 <+11073741824> log.retention.check.interval.ms=300000 zookeeper.connect=localhost:2181 zookeeper.connection.timeout.ms=6000 group.initial.rebalance.delay.ms=0 ----------------------------------------------------------------------------------------------- Regards, KAUSHIK NAMBIAR On Wed, Sep 19, 2018, 1:55 AM Liam Clarke <liam.cla...@adscale.co.nz> wrote: > Odd that the log compaction isn't working. What OS is your broker running > on and can you please post your server.properties? > > On Wed, 19 Sep. 2018, 2:13 am Kaushik Nambiar, <kaushiknambia...@gmail.com > > > wrote: > > > > > > > Hello, > > > We have a Kafka 0.11.xx version setup. > > > So the system topic which is __consumer_offset, we are looking at many > > > such topics like __consumer_offset-1,2,4..... > > > So one topic in particular,is now having log segments which is > > > contributing to 5GB of data. > > > I had a look at our server.properties file but couldnot find a property > > > called offsets.retention.minutes (found via Google). > > > So we are assuming Kafka will take the default value for such > properties. > > > So based on the default value of offset.retention.minutes property > which > > > is I guess 24 hours,but still we can see log segments older than few > > month. > > > So my question is ,is it safe to have that consumer_offset-7 topic > > (having > > > 5GB of log segments) deleted.Would it affect kafka anyway that would b > a > > > problem in production environment. > > > If deleting the topics is not a safe implementation,can you suggest a > > > different fix for this? > > > > > > > > > Thanks and Regards, > > > Kaushik Nambiar > > > > > >
