The original issue is that in windows the compaction cleanup is causing the Kafka process to crash due to file handling, in order to avoid it, I tried to disable the compaction cleanup, but that causes the consumer offsets log to keep increasing, is there a way to work with zookeeper for consumer offsets with the latest versions of Kafka and consumers, or some other way to bypass the cleanup of compacted logs ?
On Sun, Dec 30, 2018 at 8:18 PM Gwen Shapira <g...@confluent.io> wrote: > Depending on how many consumer groups and partitions you have and how often > you commit, you risk either running out of disk space and/or deleting > commit information that you will need. > Either way, you will be storing lots of records you don't need. > > Only do this if there is no other solution to whatever the real issue > is... > > On Sun, Dec 30, 2018, 5:50 PM lk gen <lkge...@gmail.com wrote: > > > Hi, > > > > the consumer offsets Kafka internal topic is always created with a > > compact cleanup policy > > > > If altering the consumer offsets topic policy from compact to delete > in a > > specific installed environment, will it cause problems ? will the > consumer > > still work if the consumer offsets are set to delete ? > > > > Thanks > > >
