Kafka Mirroring Issue: Leader Not Available Exception

I am trying to mirror a three node kafka cluster to a single node cluster. When I starts the mirror maker I got the exception LeaderNotAvailable, but at the same time topics are created and messages are commited. Please help me out. Best Regards Anshul Bhatnagar

Re: Kafka Mirroring Issue

Hi Francois, Sorry in getting back to you a bit late. I am still not able to trace the problem. No i don't have a partition key and its not respecting " topic.metadata.refresh.interval.ms" , even if i set this to 0, data is only being produced to only one partition. Ashutosh On Mon, Aug 25, 2

Re: Kafka Mirroring Issue

Do you have a partition key? IIRC, the DefaultPartitioner works differently if you use a partition key or not. If you do have a partition key, it use that algorithm: Utils.abs(key.hashCode) % numPartitions (So if your key is the same for all your topic you will always publish in the same part

Re: Kafka Mirroring Issue

I am using the default partitioner. Below are the details, source and target have auto.create.topics.enabled=true and default number of partitions is 2. bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config mirror-maker/consumer. properties --producer.config mirror-maker/producer.pr

Re: Kafka Mirroring Issue

What is your partitioning function? François Langelier Étudiant en génie Logiciel - École de Technologie Supérieure Capitaine Club Capra VP-Communication - CS Games 2014 Jeux de Génie 2011 à 2014 Magi

Re: Kafka Mirroring Issue

Hi Guozhang, I repeated my experiment with topic.metadata.refresh.interval.ms set to 60 seconds, and executed the mirror for more than 24 hours. I still see that it only writes to Partition "1" and totally ignores partition "0". Can you please provide me some pointers to debug this. Ashutosh

Re: Kafka Mirroring Issue

Hi Ashutosh, Maybe this faq can help you? https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified ? Guozhang On Thu, Aug 21, 2014 at 12:09 AM, Ashutosh Prasad Panigrahi < ashutosh1...@gmail.com> wrote: > Hello Al

Kafka Mirroring Issue

Hello All, When i start the mirroring process, events are written only to partition "1" on the target cluster and no data is written to partition 0. I am stuck on this and wondering what am i doing wrong . My production setup looks something like this 1. Datacenter1 :3 kafka nodes/3 zook