Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-14 Thread Viktor Somogyi-Vass
Hi All, It's been a few days since the last vote and we have three binding votes, so the vote passed. Thank you all who voted and participated in the discussions, I'd be excited to see this in the codebase! Binding: Jason, Harsha, Colin Non-binding: Dhruvil, Stanislav, Satish, Ryanne, Andrew, Kama

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-10 Thread Kamal Chandraprakash
+1 (non-binding). Thanks for the KIP! On Thu, Jun 6, 2019 at 8:12 PM Andrew Schofield wrote: > +1 (non-binding) > > Andrew > > On 06/06/2019, 15:15, "Ryanne Dolan" wrote: > > +1 (non-binding) > > Thanks > Ryanne > > On Wed, Jun 5, 2019, 9:31 PM Satish Duggana > wrote: > >

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-06 Thread Andrew Schofield
+1 (non-binding) Andrew On 06/06/2019, 15:15, "Ryanne Dolan" wrote: +1 (non-binding) Thanks Ryanne On Wed, Jun 5, 2019, 9:31 PM Satish Duggana wrote: > Thanks Viktor, proposed metrics are really useful to monitor replication > status on brokers.

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-06 Thread Ryanne Dolan
+1 (non-binding) Thanks Ryanne On Wed, Jun 5, 2019, 9:31 PM Satish Duggana wrote: > Thanks Viktor, proposed metrics are really useful to monitor replication > status on brokers. > > +1 (non-binding) > > On Thu, Jun 6, 2019 at 2:05 AM Colin McCabe wrote: > > > +1 (binding) > > > > best, > > Col

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-05 Thread Satish Duggana
Thanks Viktor, proposed metrics are really useful to monitor replication status on brokers. +1 (non-binding) On Thu, Jun 6, 2019 at 2:05 AM Colin McCabe wrote: > +1 (binding) > > best, > Colin > > > On Wed, Jun 5, 2019, at 03:38, Viktor Somogyi-Vass wrote: > > Hi Folks, > > > > This vote sunk a

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-05 Thread Colin McCabe
+1 (binding) best, Colin On Wed, Jun 5, 2019, at 03:38, Viktor Somogyi-Vass wrote: > Hi Folks, > > This vote sunk a bit, I'd like to draw some attention to this again in the > hope I get some feedback or votes. > > Thanks, > Viktor > > On Tue, May 7, 2019 at 4:28 PM Harsha wrote: > > > Than

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-06-05 Thread Viktor Somogyi-Vass
Hi Folks, This vote sunk a bit, I'd like to draw some attention to this again in the hope I get some feedback or votes. Thanks, Viktor On Tue, May 7, 2019 at 4:28 PM Harsha wrote: > Thanks for the kip. LGTM +1. > > -Harsha > > On Mon, Apr 29, 2019, at 8:14 AM, Viktor Somogyi-Vass wrote: > > Hi

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-05-07 Thread Harsha
Thanks for the kip. LGTM +1. -Harsha On Mon, Apr 29, 2019, at 8:14 AM, Viktor Somogyi-Vass wrote: > Hi Jason, > > I too agree this is more of a problem in older versions and therefore we > could backport it. Were you thinking of any specific versions? I guess the > 2.x and 1.x versions are defin

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-29 Thread Viktor Somogyi-Vass
Hi Jason, I too agree this is more of a problem in older versions and therefore we could backport it. Were you thinking of any specific versions? I guess the 2.x and 1.x versions are definitely targets here but I was thinking that we might not want to further. Viktor On Mon, Apr 29, 2019 at 12:5

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-28 Thread Stanislav Kozlovski
Thanks for the work done, Viktor! +1 (non-binding) I strongly agree with Jason that this monitoring-focused KIP is worth porting back to older versions. I am sure users will find it very useful Best, Stanislav On Fri, Apr 26, 2019 at 9:38 PM Jason Gustafson wrote: > Thanks, that works for me.

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-26 Thread Jason Gustafson
Thanks, that works for me. +1 By the way, we don't normally port KIPs to older releases, but I wonder if it's worth making an exception here. From recent experience, it tends to be the older versions that are more prone to fetcher failures. Thoughts? -Jason On Fri, Apr 26, 2019 at 5:18 AM Viktor

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-26 Thread Viktor Somogyi-Vass
Let me have a second thought, I'll just add the clientId instead to follow the convention, so it'll change DeadFetcherThreadCount but with the clientId tag. On Fri, Apr 26, 2019 at 11:29 AM Viktor Somogyi-Vass < viktorsomo...@gmail.com> wrote: > Hi Jason, > > Yea I think it could make sense. In t

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-26 Thread Viktor Somogyi-Vass
Hi Jason, Yea I think it could make sense. In this case I would rename the DeadFetcherThreadCount to DeadReplicaFetcherThreadCount and introduce the metric you're referring to as DeadLogDirFetcherThreadCount. I'll update the KIP to reflect this. Viktor On Thu, Apr 25, 2019 at 8:07 PM Jason Gusta

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-25 Thread Jason Gustafson
Hi Viktor, This looks good. Just one question I had is whether we may as well cover the log dir fetchers as well. Thanks, Jason On Thu, Apr 25, 2019 at 7:46 AM Viktor Somogyi-Vass wrote: > Hi Folks, > > This thread sunk a bit but I'd like to bump it hoping to get some feedback > and/or votes.

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-04-25 Thread Viktor Somogyi-Vass
Hi Folks, This thread sunk a bit but I'd like to bump it hoping to get some feedback and/or votes. Thanks, Viktor On Thu, Mar 28, 2019 at 8:47 PM Viktor Somogyi-Vass wrote: > Sorry, the end of the message cut off. > > So I tried to be consistent with the convention in LogManager, hence the > h

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-03-28 Thread Viktor Somogyi-Vass
Sorry, the end of the message cut off. So I tried to be consistent with the convention in LogManager, hence the hyphens and in AbstractFetcherManager, hence the camel case. It would be nice though to decide with one convention across the whole project, however it requires a major refactor (especia

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-03-28 Thread Viktor Somogyi-Vass
Hi Dhruvil, Thanks for the feedback and the vote. I fixed the typo in the KIP. The naming is interesting though. Unfortunately kafka overall is not consistent in metric naming but at least I tried to be consistent among the other metrics used in LogManager On Thu, Mar 28, 2019 at 7:32 PM Dhruvil

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-03-28 Thread Dhruvil Shah
Thanks for the KIP, Viktor! This is a useful addition. +1 overall. Minor nits: > I propose to add three gauge: DeadFetcherThreadCount for the fetcher threads, log-cleaner-dead-thread-count for the log cleaner. I think you meant two instead of three. Also, would it make sense to name these metrics

[VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-03-28 Thread Viktor Somogyi-Vass
Hi All, I'd like to start a vote on KIP-434. This basically would add a metrics to count dead threads in ReplicaFetcherManager and LogCleaner to allow monitoring systems to alert based on this. The KIP link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+