I am using the default partitioner. Below are the details,
source and target have auto.create.topics.enabled=true and default number of partitions is 2. bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config mirror-maker/consumer. properties --producer.config mirror-maker/producer.properties --whitelist=".*" *--num.streams 2 --num.producers 2* the consumer.properties for mirror maker looks like this group.id=test-group zookeeper.connect=<ZK_IP_LIST> and producer.properties for mirror maker looks like this metadata.broker.list=<KAFKA_IP_LIST> producer.type=async topic.metadata.refresh.interval.ms=60000 Ashutosh On Mon, Aug 25, 2014 at 6:38 PM, François Langelier <f.langel...@gmail.com> wrote: > What is your partitioning function? > > > > François Langelier > Étudiant en génie Logiciel - École de Technologie Supérieure > <http://www.etsmtl.ca/> > Capitaine Club Capra <http://capra.etsmtl.ca/> > VP-Communication - CS Games <http://csgames.org> 2014 > Jeux de Génie <http://www.jdgets.com/> 2011 à 2014 > Magistrat Fraternité du Piranha <http://fraternitedupiranha.com/> > Comité Organisateur Olympiades ÉTS 2012 > Compétition Québécoise d'Ingénierie 2012 - Compétition Senior > > > On Mon, Aug 25, 2014 at 6:18 AM, Ashutosh Prasad Panigrahi < > ashutosh1...@gmail.com> wrote: > > > Hi Guozhang, > > > > I repeated my experiment with topic.metadata.refresh.interval.ms set to > 60 > > seconds, and executed the mirror for more than 24 hours. > > > > I still see that it only writes to Partition "1" and totally ignores > > partition "0". > > > > Can you please provide me some pointers to debug this. > > > > Ashutosh > > > > > > On Thu, Aug 21, 2014 at 8:52 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > Hi Ashutosh, > > > > > > Maybe this faq can help you? > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified > > > ? > > > > > > Guozhang > > > > > > > > > On Thu, Aug 21, 2014 at 12:09 AM, Ashutosh Prasad Panigrahi < > > > ashutosh1...@gmail.com> wrote: > > > > > > > Hello All, > > > > > > > > When i start the mirroring process, events are written only to > > partition > > > > "1" on the target cluster and no data is written to partition 0. > > > > > > > > I am stuck on this and wondering what am i doing wrong . > > > > > > > > My production setup looks something like this > > > > 1. Datacenter1 : 3 kafka nodes/3 zookeeper nodes > > > > 2. Datacenter 2 : 3 kafka nodes/3 zookeeper nodes > > > > > > > > All the kafka nodes are configured with > auto.create.topics.enable=true > > > and > > > > the default partition is 2 . > > > > > > > > I start the kafka mirroring tool with the following command > > > > > > > > bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config > > > > mirror-maker/consumer.properties --producer.config > > > > mirror-maker/producer.properties --whitelist=".*" *--num.streams 2 > > > > --num.producers 2* > > > > > > > > > > > > the consumer.properties for mirror-config looks like this > > > > group.id=test-group > > > > zookeeper.connect=<ZK_IP_LIST> > > > > > > > > and producer.properties for mirror-config looks like this > > > > > > > > metadata.broker.list=<KAFKA_IP_LIST> > > > > producer.type=async > > > > topic.metadata.refresh.interval.ms=-1 > > > > > > > > > > > > > > > > Regards > > > > -- > > > > Ashutosh > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > > > -- > > Ashutosh Prasad Panigrahi > > > > All you who sleep tonight far from the ones you love > > No hands to the left or right and emptiness above > > Know that you aren't alone > > the whole world shares your tears > > some for two nights or one > > and some for all their years....... > > > -- Ashutosh Prasad Panigrahi All you who sleep tonight far from the ones you love No hands to the left or right and emptiness above Know that you aren't alone the whole world shares your tears some for two nights or one and some for all their years.......
