Hello Bret, About the properties you mentioned earlier. I couldn't find any of these properties in my server.properties file. So we were assuming the default values would b in place. So I guess the default values for the above mentioned properties are true and 24 hours for Kafka 0.11.xx
Regards, Kaushik On Wed, Sep 19, 2018, 12:04 PM Kaushik Nambiar <kaushiknambia...@gmail.com> wrote: > Hello Bret, > Thank you for your reply. > For one consumer offset topic,I can see many log segments.Each log segment > is around 100 mb. > So due to many such log segments we are experiencing such data issues. > Your views on this one > > Regards, > Kaushik Nambiar > > On Wed, Sep 19, 2018, 10:37 AM Brett Rann <br...@zendesk.com.invalid> > wrote: > >> That's unusually large. Ours are around 32k-90mb each. Initially curious >> if >> you >> have log.cleaner.enable=true and what offsets.retention.minutes is set to. >> >> And yes it can affect cluster performance. We had instances of consumer >> outages >> that were caused by bugged large consumer offfset files, especially during >> broker restarts where that partition had an index repair, or during >> leadership >> changes, correlating with a large IO spike where presumably the whole >> partition >> is read. >> >> for a given large partition, are the replicas the same size in each >> broker? >> It's >> possible for the log cleaner thread to die which needs a broker restart to >> possibly fix, and investigation if it keeps dying. Do you see any errors >> in >> your >> logs around that topic-partition? >> >> You can check (and monitor) the JMX metric for >> kafka.log:type=LogCleanerManager,name=time-since-last-run-ms which if the >> cleaner thread has died should be quite high. Our is normally around 15s. >> >> ps: https://twitter.com/gwenshap/status/1026888706429353986 >> >> On Wed, Sep 19, 2018 at 6:25 AM Liam Clarke <liam.cla...@adscale.co.nz> >> wrote: >> >> > Odd that the log compaction isn't working. What OS is your broker >> running >> > on and can you please post your server.properties? >> > >> > On Wed, 19 Sep. 2018, 2:13 am Kaushik Nambiar, < >> kaushiknambia...@gmail.com >> > > >> > wrote: >> > >> > > > >> > > > Hello, >> > > > We have a Kafka 0.11.xx version setup. >> > > > So the system topic which is __consumer_offset, we are looking at >> many >> > > > such topics like __consumer_offset-1,2,4..... >> > > > So one topic in particular,is now having log segments which is >> > > > contributing to 5GB of data. >> > > > I had a look at our server.properties file but couldnot find a >> property >> > > > called offsets.retention.minutes (found via Google). >> > > > So we are assuming Kafka will take the default value for such >> > properties. >> > > > So based on the default value of offset.retention.minutes property >> > which >> > > > is I guess 24 hours,but still we can see log segments older than few >> > > month. >> > > > So my question is ,is it safe to have that consumer_offset-7 topic >> > > (having >> > > > 5GB of log segments) deleted.Would it affect kafka anyway that >> would b >> > a >> > > > problem in production environment. >> > > > If deleting the topics is not a safe implementation,can you suggest >> a >> > > > different fix for this? >> > > > >> > > > >> > > > Thanks and Regards, >> > > > Kaushik Nambiar >> > > > >> > > >> > >> >
