Hi Guozhang, I repeated my experiment with topic.metadata.refresh.interval.ms set to 60 seconds, and executed the mirror for more than 24 hours.
I still see that it only writes to Partition "1" and totally ignores partition "0". Can you please provide me some pointers to debug this. Ashutosh On Thu, Aug 21, 2014 at 8:52 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hi Ashutosh, > > Maybe this faq can help you? > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified > ? > > Guozhang > > > On Thu, Aug 21, 2014 at 12:09 AM, Ashutosh Prasad Panigrahi < > ashutosh1...@gmail.com> wrote: > > > Hello All, > > > > When i start the mirroring process, events are written only to partition > > "1" on the target cluster and no data is written to partition 0. > > > > I am stuck on this and wondering what am i doing wrong . > > > > My production setup looks something like this > > 1. Datacenter1 : 3 kafka nodes/3 zookeeper nodes > > 2. Datacenter 2 : 3 kafka nodes/3 zookeeper nodes > > > > All the kafka nodes are configured with auto.create.topics.enable=true > and > > the default partition is 2 . > > > > I start the kafka mirroring tool with the following command > > > > bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config > > mirror-maker/consumer.properties --producer.config > > mirror-maker/producer.properties --whitelist=".*" *--num.streams 2 > > --num.producers 2* > > > > > > the consumer.properties for mirror-config looks like this > > group.id=test-group > > zookeeper.connect=<ZK_IP_LIST> > > > > and producer.properties for mirror-config looks like this > > > > metadata.broker.list=<KAFKA_IP_LIST> > > producer.type=async > > topic.metadata.refresh.interval.ms=-1 > > > > > > > > Regards > > -- > > Ashutosh > > > > > > -- > -- Guozhang > -- Ashutosh Prasad Panigrahi All you who sleep tonight far from the ones you love No hands to the left or right and emptiness above Know that you aren't alone the whole world shares your tears some for two nights or one and some for all their years.......
