On Thu, Jul 31, 2014 at 11:09:03AM -0700, Kiran Nagasubramanian wrote: > Thanks for the response Jun and Otis. > > Jun, you're right. Only the leaders are reporting. One of the brokers is > not a leader for any of the partitions. Found the following info on the > documentation page: > > leader is the node responsible for all reads and writes for the given > partition. Each node will be the leader for a *randomly* selected portion > of the partitions. > > I've been was under an assumption that the leadership for partitions is > distributed across all the brokers. *Is there a way to make all the brokers > take leadership as much evenly as possible?*
It is distributed, although it may not be an optimal distribution. i.e., as long as you have more than just a handful of topics the leader counts across the brokers should be relatively even. If you do a rolling bounce of the cluster the leader counts can get skewed in which case you can run a preferred replica leader election operation: http://kafka.apache.org/documentation.html#basic_ops_leader_balancing > > We're using a custom app for metrics collection. > > Otis, we're using kafka_2.9.2-0.8.1 > > Thanks. > > > > On Thu, Jul 31, 2014 at 6:26 AM, Otis Gospodnetic < > [email protected]> wrote: > > > Hi, > > > > May also want to share version of Kafka. We have SPM agents for Kafka > > 0.7.x and 0.8.x and we haven't seen that. > > > > Otis > > -- > > Performance Monitoring * Log Analytics * Search Analytics > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > On Wed, Jul 30, 2014 at 10:12 PM, Kiran Nagasubramanian < > > [email protected]> > > wrote: > > > > > Hello, > > > > > > We're collecting the JMX metrics from the Kafka brokers. We're seeing a > > > couple of issues. Could someone please throw some light if you've come > > > across something similar? > > > > > > 1) We have a 3 broker Kafka cluster and when we're trying to collect the > > > metrics like messages in per sec, bytes in per sec, etc. we get the > > values > > > as 0 for one of the three brokers. But we get proper values for metrics > > > like heap memory usage for all the brokers. When we restart the cluster, > > > the same or some other broker would behave in a similar way. > > > > > > We're seeing similar behavior in another cluster as well. > > > > > > 2) We're logging the time it takes to collect the metrics. The time to > > > collect seems to increase over time and crosses a minute in a couple of > > > days. It's of the order of 1 or 2 seconds when start the cluster. > > > > > > Thanks. > > > > >
