Re: Kafka Stream: State replication seems unpredictable.

2021-04-22 Thread Parthasarathy, Mohan
Guozhang, What does this mean if the changelog topic was disabled ? If thread 2 and thread 4 are running in two different nodes and a rebalance occurs, thread 2 will not realize it is a zombie without the write to the changelog topic, right ? I am trying to understand the cases under which the

Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 2.8.0

2021-04-22 Thread Colin McCabe
Hi Ran, As part of the KRaft work outlined in KIP-500, we are planning on creating an authorizer that does not rely on ZooKeeper. This work is not included in the 2.8 release, however. regards, Colin On Mon, Apr 19, 2021, at 13:02, Ran Lupovich wrote: > Hello, Maybe I missed it in the docume

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

2021-04-22 Thread Colin McCabe
Congratulations, Randall! best, Colin On Wed, Apr 21, 2021, at 09:51, Mickael Maison wrote: > Congratulations Randall, well deserved! > > On Mon, Apr 19, 2021 at 10:55 PM Konstantine Karantasis > wrote: > > > > Congratulations Randall! > > > > Konstantine > > > > On Mon, Apr 19, 2021 at 1:14 A

Kafka Connect Dist. Worker Does not join group

2021-04-22 Thread Upesh Desai
Hello, Has anyone experienced this scenario before when having a distributed connect cluster: [kafka-coordinator-heartbeat-thread | connect-cluster] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Worker clientId=connect-1, groupId=connect-cluster] Member connect-1-50ee

Kafka Connect: producer idempotence after increasing producer.max.in.flight.requests.per.connection

2021-04-22 Thread Yong Gang
Hi guys, We are using Kafka Connect to do Kafka to Kafka data replication between DCs. For this config item *producer.max.in.flight.requests.per.connection* the default value in KafkaProducer (Kafka client) is 5 and can still maintain idempotence, but in Kafka Connect the default value of this is

mirrormaker 2.0

2021-04-22 Thread Dhanikachalam, Madhan (CORP)
I am testing MM2. I got the connector working but it is creating topics in the downstream cluster like this mm-poc-src.grp1-top1 which is the alias.. How can I create the downstream topic to be the exact name as the source? Also, do you provide commercial support for open source kafka? Or direc

Re: Kafka Connect: producer idempotence after increasing producer.max.in.flight.requests.per.connection

2021-04-22 Thread Liam Clarke-Hutchinson
Hi Yong Gang, Idempotence is not guaranteed if you increase max in flight requests. Because if 5 requests are dispatched, but 2 requests fail and are retried, the messages in those 2 requests will arrive after the messages in the other three requests, so ordering will be changed. Assuming everyth