MirrorMaker2 is not always starting tasks

2021-08-17 Thread Jozef Vilcek
I am using MirrorMaker 2.0 and running it via MirrorMaker.java class. This method will start up it's own worker and`DistributedHerder`. After upgrading from version `2.4.0` to

Re: DescribeTopics could return deleted topic

2021-08-17 Thread Guozhang Wang
Since DescribeTopics is just handled via a Metadata request behind the scene, it is possible that if the request is sent to some brokers with stale metadata (not yet received the metadata update request). But I would not expect it to "always" return the deleted topic partition, unless the broker be

Re: High disk read with Kafka streams

2021-08-17 Thread Guozhang Wang
Hello Magnat, Thanks for reporting your observations. I have some questions: 1) Are your global state stores also in-memory or persisted on disks? 2) Are your Kafka and KStreams colocated? Guozhang On Tue, Aug 10, 2021 at 6:10 AM mangat rai wrote: > Hey All, > > We are using the low level pr

Re: DescribeTopics could return deleted topic

2021-08-17 Thread Boyang Chen
Got it, thanks for explaining that Guozhang! On Tue, Aug 17, 2021 at 3:08 PM Guozhang Wang wrote: > Since DescribeTopics is just handled via a Metadata request behind the > scene, it is possible that if the request is sent to some brokers with > stale metadata (not yet received the metadata upda

Kafka-consumer-groups.sh says "no active members" but the CURRENT_OFFSET is moving

2021-08-17 Thread Mayuresh Gharat
Hi Folks, We recently came across a weird scenario where we had a consumer group consuming from multiple topics. When we ran the "Kafka-consumer-group" command multiple times, we saw that the CURRENT-OFFSET is advancing; however , we also saw a line printed: *"Consumer group 'GROUP_ABC' has no act