Thanks very much for the reply Manikumar! I found that there were a few topics on the source cluster that had more than two partitions, but all topics on the target cluster had 2 partitions. I did a test between one topic that had 2 on each, and I did get messages to both partitions as expected.
Is there a way to get this same behavior if the number of partitions on the source and target topic are different? What should I set the num.consumer.fetchers property to? I believe I remember reading that it needed to be at least equal to the number of partitions on the topic. For example, if I'm reading from: topicA: 2 partitions topicB: 3 partitions topicC: 1 partition should i set num.consumer.fetchers to 3, or 6? :) Again, thank you!! On Fri, Mar 24, 2017 at 12:57 AM, Manikumar <manikumar.re...@gmail.com> wrote: > Are you sure target cluster topics have more than one partition? > If you are sending keyed messages, they may be going to the same partition. > > On Thu, Mar 23, 2017 at 11:15 PM, Chris Neal <cwn...@gmail.com> wrote: > > > Hi everyone, > > > > I am using MirrorMaker to consume from a 0.8.2.2 cluster and produce to a > > 0.10.2 cluster. All the topics have two partitions on both clusters. My > > consumer.properties is: > > > > zookeeper.connect=[string of servers] > > group.id=MirrorMaker > > num.consumer.fetchers=2 > > partition.assignment.strategy=roundrobin > > > > My producer.properties is: > > > > metadata.broker.list=[string of servers] > > compression.codec=gzip > > producer.type=async > > message.send.max.retries=2 > > > > My understanding from the documentation is that this *should* work, but > it > > is not. All my data is going into partition 1 on the target cluster for > > all topics. > > > > Can someone help me understand what I'm missing here? > > > > Thank you for your time! > > Chris > > >
