Re: GlobalKTable restoration - Unexplained performance penalty

Hey, First of all I want to apologize for thinking our own implementation of StateRestoreListener was a default log (got confused there). This is our implementation (which shows its not really the # of object, just the diff in offsets) > override def onRestoreStart(topicPartition: TopicPartition,

Previous State of a Kafka Message

Hi, Greetings! My requirement is as below. I have Topic named "sample-topic". This topic contains the same keys(as String) with multiple messages and the messages are in JSON format. I would like to merge the JSON messages and produce a final JSON. In order to achieve this, how to maintain the

Re: Previous State of a Kafka Message

Hi Suresh, It seems the keys are different for each message - I see 0, 1 and 2. When you say "This topic contains the same keys" do you mean the message contents contain the same "attr1" "attr2" "attr3" keys? If so, I think the first step would be to re-key the messages so the new key is "attr1:at

Re: Maintaining same offset while migrating from Confluent Replicator to Apache Mirror Maker 2.0

Hi Amit, I guess you may need to override the actual consumer group config (probably not consumer.group.id) that is used in Kafka Connect  On 2020/11/26 06:47:11, wrote: > Hi All, > > We are currently trying to migrate Confluent replicator to Apache Open Source > Mirror Maker v2.0. We are fac

RE: Maintaining same offset while migrating from Confluent Replicator to Apache Mirror Maker 2.0

Hi Ning, Thank you for the response. I probably tried to change every possible consumer group id in all ways MM2 can run. But I went through the code and it seems consumer config is using assign method which does not requires group id to be assigned and also could not find anywhere that proper

Re: [VOTE] 2.7.0 RC4

Hey Bill, Unfortunately we have found another regression in 2.7 streams, which I have filed a blocker here . The implementation is done, and I will try to get reviews and merge ASAP. Best, Boyang On Fri, Dec 4, 2020 at 3:14 PM Jack Yang wrote:

Kafka Streams Optimizations

Hi All, I have been working on moving an application to kafka streams and I have the following questions. 1. We were planning to use an EFS mount to share rocksdb data for KV store and global state store with which we were hoping to minimize the state restore time when new instances are brought u