Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

I'd like to bump the thread to get review on the PR. https://github.com/apache/kafka/pull/6843 Thanks, all! On Wed, Jul 3, 2019 at 9:33 AM Cyrus Vafadari wrote: > Thanks for the input, everyone! I'm marking this KIP as accepted with > 3 binding votes (Gwen, Guozhang, Randall) > 3 nonbinding vot

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Thanks for the input, everyone! I'm marking this KIP as accepted with 3 binding votes (Gwen, Guozhang, Randall) 3 nonbinding votes (Andrew, Konstantine, Ryanne) The PR is updated at https://github.com/apache/kafka/pull/6843/files On Sun, Jun 23, 2019 at 10:52 AM Andrew Schofield wrote: > +1 (n

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

+1 (non-binding). Nice KIP. On 23/06/2019, 17:27, "Gwen Shapira" wrote: +1 from me too On Tue, Jun 18, 2019, 10:30 AM Guozhang Wang wrote: > Cyrus, thanks for the updates, +1. > > On Mon, Jun 17, 2019 at 10:58 PM Cyrus Vafadari > wrote: > > > Many tha

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

+1 from me too On Tue, Jun 18, 2019, 10:30 AM Guozhang Wang wrote: > Cyrus, thanks for the updates, +1. > > On Mon, Jun 17, 2019 at 10:58 PM Cyrus Vafadari > wrote: > > > Many thanks for the feedback. Per Gwen's suggestion, I've updated the KIP > > to specify that the task count will be per-wor

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Cyrus, thanks for the updates, +1. On Mon, Jun 17, 2019 at 10:58 PM Cyrus Vafadari wrote: > Many thanks for the feedback. Per Gwen's suggestion, I've updated the KIP > to specify that the task count will be per-worker (no additional MBean tag, > since each process is a worker) and per-connector

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Many thanks for the feedback. Per Gwen's suggestion, I've updated the KIP to specify that the task count will be per-worker (no additional MBean tag, since each process is a worker) and per-connector (MBean tag). On Mon, Jun 17, 2019 at 8:24 PM Cyrus Vafadari wrote: > I meant to write: > I've al

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

I meant to write: I've also updated the KIP to clarify that every task must have exactly one non-null *status* at all times. On Mon, Jun 17, 2019 at 6:55 PM Cyrus Vafadari wrote: > Guozhang, > > Both of Kafka's implementations of "StatusBackingStore" immediately delete > the task from the backig

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Guozhang, Both of Kafka's implementations of "StatusBackingStore" immediately delete the task from the backign store when you try to set it to DESTROYED, so we'd actually expect it to always be zero. A nonzero number of destroyed tasks would either indicate a new implementation of StatusBackingSto

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Hello Cyrus, Thanks for the KIP. I just have one nit question about Connect destroyed tasks: is it an ever-increasing number? If yes, the corresponding metric value would be increasing indefinitely as well. Is that intentional? Otherwise, lgtm. Guozhang On Mon, Jun 17, 2019 at 1:14 PM Gwen Sha

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Sorry to join so late, but did we consider a single set of task-count metrics and using tags to scope each data point to a specific connector and worker (and in the future perhaps also user)? It will make analysis of the data easier - someone may want to breakdown tasks by both worker and connecto

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Konstantine, This is a good suggestion. Since the suggestion to add 2 additional statuses analogous to the 3 proposed, it is a very minor change of no structural consequence to the KIP. I've updated the KIP to incorporate your suggestion, and any voters who disagree should definitely respond in t

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Thanks Cyrus, this is a nice and straightforward addition. I'm +1 too, but I'd like to return with a question here as well regarding whether the unassigned tasks will be taken into account. Especially after KIP-415 we might start seeing this status for specific periods of time. Therefore, I think

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Thanks, Cyrus. +1 (binding) Randall Hauch On Wed, Jun 5, 2019 at 10:36 AM Andrew Schofield wrote: > +1 (non-binding) > > Andrew Schofield > > On 05/06/2019, 14:04, "Ryanne Dolan" wrote: > > +1 (non-binding) > > Thanks > Ryanne > > On Tue, Jun 4, 2019, 11:29 PM Cyrus Vafadari

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

+1 (non-binding) Andrew Schofield On 05/06/2019, 14:04, "Ryanne Dolan" wrote: +1 (non-binding) Thanks Ryanne On Tue, Jun 4, 2019, 11:29 PM Cyrus Vafadari wrote: > Hi all, > > Like like to start voting in the following KIP: > > https://nam01.

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

+1 (non-binding) Thanks Ryanne On Tue, Jun 4, 2019, 11:29 PM Cyrus Vafadari wrote: > Hi all, > > Like like to start voting in the following KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector > > Discussion thread: > > https

[VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Hi all, Like like to start voting in the following KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector Discussion thread: https://lists.apache.org/thread.html/bf7c92224aa798336c14d7e96ec8f2e3406c61879ec381a50652acfe@%3Cdev.kafka.a