>From memory, MirrorMaker is just using Kafka Producers and Consumers to send the data from one DC to the other. So the meaningful performance metric I would be looking at is how far behind your mirror queues are from your source queues. Your other performance metrics are going to be very dependent on what's going on in the source queue.
For stress testing, I would create a constant source of Kafka messages, and increase the rate by 50% every 10 minutes until the MM can no longer keep up. Your network link throughput is almost certainly going to be the bottleneck. On Wed, 22 Apr 2015 at 8:30 am Otis Gospodnetic <otis.gospodne...@gmail.com> wrote: > Hi, > > I don't know much about MM yet, but what sort of MM performance are you > looking for? If MM acts as just another producer for another Kafka cluster > (just a guess!) then you could look at the metrics of the consuming cluster > to see how fast it's ingesting.... but if MM works differently, then > something else is needed. > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > > > On Tue, Mar 17, 2015 at 5:46 PM, Saladi Naidu > <naidusp2...@yahoo.com.invalid > > wrote: > > > Any suggestions on how to measure throughput of the Mirror Maker Naidu > > Saladi > > > > ----- Forwarded Message ----- > > From: Saladi Naidu <naidusp2...@yahoo.com> > > To: "users@kafka.apache.org" <users@kafka.apache.org> > > Sent: Monday, March 16, 2015 10:31 PM > > Subject: How to measure performance of Mirror Maker > > > > We have three Kafka clusters deployed in 3 DC's, each one having their > own > > topics. We are using Mirror Maker to keep all the three clusters up to > date > > with continuous replication using Mirror Maker. We used Perf producer > and > > Perf consumer to conduct basic testing, all seems to work great. > > Our 2 DC's are in US, 50 miles a part well connected by MAN fiber, our > 3rd > > DC is in Europe, which is connected with dedicated links to 2 DC's in US, > > round trip latency is around 300 ms. > > When we ran performance test, our client node where perf test scripts ran > > are in US DC's, I have observed very low through put (almost 1/100th), I > > understand some part is due to NW latency. Same will be applied to > Mirror > > Maker as well right? > > Are there any tools to measure Mirror Maker performance in general and > > specifically in remote DC? Naidu Saladi > > > > > > > > >
