Hi Clark My source cluster is on localhost:9092
My Target cluster was localhost:9091 I am not aware of something like Mirror maker 1 and 2 Because I am using mirror maker which come default from Apache Kafka 2.3.0 On Thu, Apr 30, 2020, 09:43 Liam Clarke-Hutchinson < liam.cla...@adscale.co.nz> wrote: > Hi Vishnu, > > I can see you're using Mirror Maker 1, unlike Mirror Maker 2, this doesn't > auto-create topics on the target cluster. Does your target cluster on port > 9091 have the same topics created on it that exist on the source cluster on > port 9092? > > Also note that a Kafka consumer defaults to "latest" for auto offset reset > - so when starting MM 1 it won't automatically replicate all existing data, > only data created after it begins consuming. You can override this by > setting the consumer property "auto.offset.reset" to "earliest". > > Hope that helps, > > Liam Clarke-Hutchinson > > > On Thu, Apr 30, 2020 at 3:56 PM vishnu murali <vishnumurali9...@gmail.com> > wrote: > > > i start the kafka mirror using the below configuration > > > > .\bin\windows\kafka-mirror-maker.bat --consumer.config > > .\config\consumer.properties --producer.config > .\config\producer.properties > > --whitelist=".*" > > > > *Consumer.properties:* > > # format: host1:port1,host2:port2 ... > > bootstrap.servers=localhost:9092 > > > > # consumer group id > > group.id=test-consumer-group > > > > *Producer.properties:* > > # format: host1:port1,host2:port2 ... > > bootstrap.servers=localhost:9091 > > > > # specify the compression codec for all data generated: none, gzip, > snappy, > > lz4, zstd > > compression.type=none > > > > > > I get this kind of warning statement and it will be still present in > that > > statement of more than 30 minutes.No data will be transfered > > > > WARNING: The default partition assignment strategy of the mirror maker > will > > change from 'range' to 'roundrobin' in an upcoming release (so that > better > > load balancing can be achieved). If you prefer to make this switch in > > advance of that release add the following to the corresponding config: > > > > > 'partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor' > > > > can anyone clarify why i am getting this and what i am doing wrong? > > > > On Thu, Apr 30, 2020 at 12:22 AM vishnu murali < > vishnumurali9...@gmail.com > > > > > wrote: > > > > > Thanks Blake..😊😊 > > > > > > More over can we use any connector types for this situation? > > > > > > Like source is a topic and sink is also an another topic > > > > > > Is this possible... > > > > > > On Thu, Apr 30, 2020, 00:19 Blake Miller <blak3mil...@gmail.com> > wrote: > > > > > >> Hi Vishnu, > > >> > > >> Check out MirrorMaker > > >> > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330 > > >> > > >> This can do what you want. Note that the offsets are not copied, nor > are > > >> the message timestamps. > > >> > > >> HTH > > >> > > >> > > >> On Wed, Apr 29, 2020 at 6:47 PM vishnu murali < > > vishnumurali9...@gmail.com > > >> > > > >> wrote: > > >> > > >> > Hi Guys, > > >> > > > >> > I am having two separate Kafka cluster running in two independent > > >> zookeeper > > >> > > > >> > I need to send a set of data from one topic from cluster A to > cluster > > B > > >> > with the same topic name with all data also.. > > >> > > > >> > How can I achieve this > > >> > > > >> > Done anyone have any idea ?? > > >> > > > >> > > > > > >
