Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-17 Thread Viktor Somogyi-Vass
Hi Ismael, By Jason's suggestion we finally went with the originally voted proposal that is to include reassignment bytes in/out in replication bytes in/out and we discuss this when throttling is changed. Sorry for not updating this thread, I was busy with the code review. I'll get the KIP in shape

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-16 Thread Ismael Juma
The current proposal says that replication throughput would change not to include reassignment though. Ismael On Wed, Oct 16, 2019, 11:53 AM Colin McCabe wrote: > Hi Ismael, > > I think every replica is doing replication, by definition. But not every > replica is undergoing reassignment. > > I

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-16 Thread Colin McCabe
Hi Ismael, I think every replica is doing replication, by definition. But not every replica is undergoing reassignment. If the broker that died was in the set of new replicas being added, its death will not add a new under-replicated partition. Otherwise, it will add a new URP. best, Colin

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-16 Thread Ismael Juma
If a broker dies and loses the disk, is it replication or reassignment? Ismael On Thu, Oct 10, 2019 at 3:30 AM Viktor Somogyi-Vass wrote: > Hi All, > > During the code review it came up that we shouldn't count replication bytes > together with reassignment bytes so they count to a different met

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-11 Thread Stanislav Kozlovski
+1 (non-binding), thanks for bringing it up On Thu, Oct 10, 2019 at 5:48 PM Colin McCabe wrote: > +1. Thanks, Viktor. > > Colin > > On Thu, Oct 10, 2019, at 03:30, Viktor Somogyi-Vass wrote: > > Hi All, > > > > During the code review it came up that we shouldn't count replication > bytes > > to

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-10 Thread Colin McCabe
+1. Thanks, Viktor. Colin On Thu, Oct 10, 2019, at 03:30, Viktor Somogyi-Vass wrote: > Hi All, > > During the code review it came up that we shouldn't count replication bytes > together with reassignment bytes so they count to a different metrics. This > is a change in the semantics of Replicat

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-10-10 Thread Viktor Somogyi-Vass
Hi All, During the code review it came up that we shouldn't count replication bytes together with reassignment bytes so they count to a different metrics. This is a change in the semantics of ReplicationBytesInPerSec and ReplicationBytesOutPerSec metrics but since we plan to separate reassignment

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-26 Thread Jason Gustafson
Closing this vote. The final result is +9 with 4 binding votes. @Satish Sorry, I missed your question above. Good point about updating documentation. I will create a separate jira to make sure this gets done. -Jason On Fri, Aug 23, 2019 at 11:23 AM Jason Gustafson wrote: > Thanks Stan, good ca

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Jason Gustafson
Thanks Stan, good catch. I have updated the KIP. I will plan to close the vote Monday if there are no objections. -Jason On Fri, Aug 23, 2019 at 11:14 AM Colin McCabe wrote: > On Fri, Aug 23, 2019, at 11:08, Stanislav Kozlovski wrote: > > Thanks for the KIP, this is very helpful > > > > I had a

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Colin McCabe
On Fri, Aug 23, 2019, at 11:08, Stanislav Kozlovski wrote: > Thanks for the KIP, this is very helpful > > I had an offline discussion with Jason and we discussed the semantics of > the underMinIsr/atMinIsr metrics. The current proposal would expose a gap > where we could report URP but no MinIsr.

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Stanislav Kozlovski
Thanks for the KIP, this is very helpful I had an offline discussion with Jason and we discussed the semantics of the underMinIsr/atMinIsr metrics. The current proposal would expose a gap where we could report URP but no MinIsr. A brief example: original replica set = [0,1,2] new replica set = [3,

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Colin McCabe
+1 (binding). cheers, Colin On Tue, Aug 20, 2019, at 10:55, Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to > fix > a long-known shortcoming of URP reporting and to improve reassignment > monitoring: > https://cwiki.apache.org/conflue

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-22 Thread Vahid Hashemian
+1 (binding) Thanks Jason. This is super useful. --Vahid On Tue, Aug 20, 2019 at 10:55 AM Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to fix > a long-known shortcoming of URP reporting and to improve reassignment > monitoring: > > h

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-22 Thread Harsha Ch
+1 (binding) Thanks, Harsha On Wed, Aug 21, 2019 at 4:23 PM, Robert Barrett < bob.barr...@confluent.io > wrote: > > > > +1 (non-binding) > > > > This will be great to have, thanks Jason! > > > > On Wed, Aug 21, 2019 at 4:29 AM Manikumar < manikumar. reddy@ gmail. com ( > manikumar.re.

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-21 Thread Robert Barrett
+1 (non-binding) This will be great to have, thanks Jason! On Wed, Aug 21, 2019 at 4:29 AM Manikumar wrote: > +1 (binding). > > Thanks for the KIP. LGTM. > > > On Wed, Aug 21, 2019 at 3:12 PM Satish Duggana > wrote: > > > Hi Jason, > > +1 (non binding) Thanks for the KIP! > > > > Do we need to

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-21 Thread Manikumar
+1 (binding). Thanks for the KIP. LGTM. On Wed, Aug 21, 2019 at 3:12 PM Satish Duggana wrote: > Hi Jason, > +1 (non binding) Thanks for the KIP! > > Do we need to have a separate JIRA to update the docs as it introduces new > metrics and a change in behavior for the existing metric? > > > > On

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-21 Thread Satish Duggana
Hi Jason, +1 (non binding) Thanks for the KIP! Do we need to have a separate JIRA to update the docs as it introduces new metrics and a change in behavior for the existing metric? On Wed, Aug 21, 2019 at 2:41 PM Mickael Maison wrote: > +1 (non binding) > Thanks Jason > > On Wed, Aug 21, 2019

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-21 Thread Mickael Maison
+1 (non binding) Thanks Jason On Wed, Aug 21, 2019 at 8:15 AM David Jacot wrote: > > +1 (non-binding) > > Thanks for the KIP! > > Best, > David > > On Tue, Aug 20, 2019 at 7:55 PM Jason Gustafson wrote: > > > Hi All, > > > > I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-21 Thread David Jacot
+1 (non-binding) Thanks for the KIP! Best, David On Tue, Aug 20, 2019 at 7:55 PM Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to fix > a long-known shortcoming of URP reporting and to improve reassignment > monitoring: > > https://cw

[VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-20 Thread Jason Gustafson
Hi All, I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to fix a long-known shortcoming of URP reporting and to improve reassignment monitoring: https://cwiki.apache.org/confluence/display/KAFKA/KIP-352%3A+Distinguish+URPs+caused+by+reassignment . Note that I have added one n