Re: NetworkProcessorAvgIdlePercent

2015-03-02 Thread Zakee
Thanks, I have added them for monitoring. -Zakee > On Feb 27, 2015, at 9:21 AM, Jun Rao wrote: > > Zakee, > > It would be useful to get the following. > > kafka.network:name=RequestQueueSize,type=RequestChannel > kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics > kaf

Re: NetworkProcessorAvgIdlePercent

2015-02-27 Thread Jun Rao
Zakee, It would be useful to get the following. kafka.network:name=RequestQueueSize,type=RequestChannel kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics Thanks, Jun On Thu, Feb 26, 2015 at 2:17 PM

Re: NetworkProcessorAvgIdlePercent

2015-02-26 Thread Zakee
Hi Jun, There are too many KPIs generated by kafka that I can’t monitor them all, I have monitoring a filtered list that I can understand so far. Which keys are you talking of? Currently I am not monitoring any of below keys but will add only those that are useful. kafka.network:name=RequestQu

Re: NetworkProcessorAvgIdlePercent

2015-02-26 Thread Jun Rao
That may be enough. What's the RequestQueueSize and RequestQueueTimeMs? Thanks, Jun On Wed, Feb 25, 2015 at 10:24 PM, Zakee wrote: > Well currently I have configured 14 thread for both io and network. Do you > think we should consider more? > > Thanks > -Zakee > > On Wed, Feb 25, 2015 at 6:22

Re: NetworkProcessorAvgIdlePercent

2015-02-25 Thread Zakee
Well currently I have configured 14 thread for both io and network. Do you think we should consider more? Thanks -Zakee On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao wrote: > Then you may want to consider increasing num.io.threads > and num.network.threads. > > Thanks, > > Jun > > On Tue, Feb 24, 20

Re: NetworkProcessorAvgIdlePercent

2015-02-25 Thread Jun Rao
Then you may want to consider increasing num.io.threads and num.network.threads. Thanks, Jun On Tue, Feb 24, 2015 at 7:48 PM, Zakee wrote: > Similar pattern for that too. Mostly hovering below. > > -Zakee > > On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao wrote: > > > What about RequestHandlerAvgId

Re: NetworkProcessorAvgIdlePercent

2015-02-24 Thread Zakee
Similar pattern for that too. Mostly hovering below. -Zakee On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao wrote: > What about RequestHandlerAvgIdlePercent? > > Thanks, > > Jun > > On Mon, Feb 23, 2015 at 8:47 PM, Zakee wrote: > > > Hi Jun, > > > > With ~100G of data being pushed per hour across 35

Re: NetworkProcessorAvgIdlePercent

2015-02-24 Thread Jun Rao
What about RequestHandlerAvgIdlePercent? Thanks, Jun On Mon, Feb 23, 2015 at 8:47 PM, Zakee wrote: > Hi Jun, > > With ~100G of data being pushed per hour across 35 topics > (replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly > showing below 0.5 sometimes when the producers sen

Re: NetworkProcessorAvgIdlePercent

2015-02-23 Thread Zakee
Hi Jun, With ~100G of data being pushed per hour across 35 topics (replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly showing below 0.5 sometimes when the producers send on a high rate. Thanks -Zakee On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao wrote: > What kind of load do you h

Re: NetworkProcessorAvgIdlePercent

2015-02-22 Thread Jun Rao
What kind of load do you have on the brokers? On an idle cluster (just fetch requests from the follower replicas), I saw NetworkProcessorAvgIdlePercent at about 97%. Thanks, Jun On Thu, Feb 19, 2015 at 5:19 PM, Zakee wrote: > Jun, > > I am already using the latest release 0.8.2.1. > > -Zakee >

Re: NetworkProcessorAvgIdlePercent

2015-02-19 Thread Zakee
Jun, I am already using the latest release 0.8.2.1. -Zakee On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao wrote: > Could you try the 0.8.2.1 release being voted on now? It fixes a CPU issue > and should reduce the CPU load in network thread. > > Thanks, > > Jun > > On Thu, Feb 19, 2015 at 11:54 AM,

Re: NetworkProcessorAvgIdlePercent

2015-02-19 Thread Jun Rao
Could you try the 0.8.2.1 release being voted on now? It fixes a CPU issue and should reduce the CPU load in network thread. Thanks, Jun On Thu, Feb 19, 2015 at 11:54 AM, Zakee wrote: > Kafka documentation recommends <0.3 for above metric. I assume processor is > busier if this goes below 0.3