Kafka consumer ensure offsets committed before partition rebalance to avoid duplicates

2022-09-14 Thread Pushkar Deole
Hi All, I am hosting kafka consumers inside microservice hosted as kubernetes pods, 3 consumers in a consumer group. There is a requirement to add auto-scaling where there will be a single pod which will be auto-scaled out or scaled-in based on the load on microservice. So, 1 pod can be scaled out

Re: Kafka stream moving from running to shutdown without any visible exception or error

2022-09-14 Thread Bruno Cadonna
Hi Samuel, from your few log messages I see that the global stream thread dies. That might be the cause of the shutdown. Look for what is causing your global stream thread/global store to die. Best, Bruno On 13.09.22 22:55, Samuel Azcona wrote: Hi guys, I have a kotlin App that run 5 str

Mirror maker disable auto topic creation

2022-09-14 Thread Mcs Vemuri
Hello, Is there any way to disable topic creation in MM2? I tried setting the topic.creation.enable to False but the MM-offsets/configs/status topics are still created(broker auto topic creation is also set to false) I found this- https://groups.google.com/g/confluent-platform/c/JRZmpCEZElo  whi

Re: Mirror maker disable auto topic creation

2022-09-14 Thread Ryanne Dolan
The link is correct. Connect and mm2 create those internal topics at startup, whether or not auto topic creation is enabled. On Wed, Sep 14, 2022, 11:15 PM Mcs Vemuri wrote: > Hello, > Is there any way to disable topic creation in MM2? I tried setting the > topic.creation.enable to False but the