Re: Kafka Mirror maker stops replicating

2023-02-23 Thread Arpit Jain
I tried again by increasing the max.poll.interval.ms to a really high number like 12 hrs and since then it has been replicating fine. What I observed was that after every max.poll.interval elapse, it was getting stuck and replication was stopping. Putting a high number like 12 hrs, it worked fine f

Re: Kafka Mirror maker stops replicating

2023-02-08 Thread Greg Harris
Arpit, I am not very familiar with MirrorMaker unfortunately so I won't be able to give you any specific advice. I also don't see any MirrorMaker-specific changes that would be relevant, except for some minor arguments changes and the deprecation landing in 3.0. > Its very random. It replicates f

Re: Kafka Mirror maker stops replicating

2023-02-08 Thread Arpit Jain
Hi Greg, Thanks for getting back to me. Please find more details below 1. Are you using MirrorMaker, or MirrorMaker 2.0? Mirror maker 2. What version of MM or MM2 are you using, and with what Kafka broker version? 3.2.3 3. How is your replication flow configured? We have upstream brokers (3 node

Re: Kafka Mirror maker stops replicating

2023-02-08 Thread Greg Harris
Arpit, Unfortunately from that description nothing specific is coming to mind. The max.poll.interval indicates that the consumer is losing contact with the Kafka cluster, but that may be caused by the replication application hanging somewhere else. Some clarifying questions, and things you can lo