Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-05-26 Thread Randall Hauch
Sorry, I missed Matthias' +1 binding. I'll move the KIP back to "Adopted" and add it to the AK 2.6.0. Apologies for the noise. On Tue, May 26, 2020 at 12:14 PM Randall Hauch wrote: > Just a quick note: I've changed > https://cwiki.apache.org/confluence/display/KAFKA/KIP-557%3A+Add+emit+on+chang

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-05-26 Thread Randall Hauch
Just a quick note: I've changed https://cwiki.apache.org/confluence/display/KAFKA/KIP-557%3A+Add+emit+on+change+support+for+Kafka+Streams to denote that this KIP is still in voting, as it has only received 2 binding votes. I will also remove the KIP from the AK 2.6.0 release, since the KIP freeze (

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-07 Thread Richard Yu
Hi Matthias, Oh, I see. Next time, I will take that into account. It looked like at the time there wasn't much contention over the major points of the proposal, so I thought I could pass it. I will also make some last modifications to the KIP. Thanks for your vote! Best, Richard On Sat, Mar 7

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-07 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Richard, you cannot close a KIP as accepted with 2 binging votes. (cf https://cwiki.apache.org/confluence/display/KAFKA/Bylaws) You could only discard the KIP as long as it's not accepted :D However, I am +1 (binding) and thus you can close the VO

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-06 Thread Richard Yu
Hi all, I have decided to pass this KIP with 2 binding votes and 3 non-binding votes (including mine). I will update KIP status shortly after this. Best, Richard On Thu, Mar 5, 2020 at 3:45 PM Richard Yu wrote: > Hi all, > > Just polling for some last changes on the name. > I think that since

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-05 Thread Richard Yu
Hi all, Just polling for some last changes on the name. I think that since there doesn't seem to be much objection to any major changes in the KIP, I will pass it this Friday. If you feel that we still need some more discussion, please let me know. :) Best, Richard P.S. Will start working on a

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-04 Thread Guozhang Wang
Regarding the metric name, I was actually trying to be consistent with the node-level `suppression-emit` as I feel this one's characteristics is closer to that. I other folks feels better to align with the task-level "dropped-records" I think I can be convinced too. Guozhang On Wed, Mar 4, 2020

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-04 Thread Bruno Cadonna
Hi all, may I make a non-binding proposal for the metric name? I would prefer "skipped-idempotent-updates" to be consistent with the "dropped-records". Best, Bruno On Tue, Mar 3, 2020 at 11:57 PM Richard Yu wrote: > > Hi all, > > Thanks for the discussion! > > @Guozhang, I will make the corresp

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-03 Thread Richard Yu
Hi all, Thanks for the discussion! @Guozhang, I will make the corresponding changes to the KIP (i.e. renaming the sensor and adding some notes). With the current state of things, we are very close. Just need that one last binding vote. @Matthias J. Sax It would be ideal if we can also get your

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-03 Thread Guozhang Wang
Hi Bruno, John: 1) That makes sense. If we consider them to be node-specific metrics that only applies to a subset of built-in processor nodes that are irrelevant to alert-relevant metrics (just like suppression-emit (rate | total)), they'd better be per-node instead of per-task and we would not a

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-03 Thread John Roesler
Thanks, Guozhang and Bruno! 2) I had a similar though to both of you about the metrics, but I ultimately came out with a conclusion like Bruno's. These aren't dropped invalid records, they're intentionally dropped, valid, but unnecessary, updates. A "warning" for this case definitely seems wrong,

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-03 Thread Bruno Cadonna
Hi Guozhang, I also had the same thought about using the existing "dropped-records" metrics. However, I think in this case it would be better to use a new metric because dropped idempotent updates is an optimization, they do not represent missed records. The dropped idempotent updates in general d

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-02 Thread Guozhang Wang
Hello Richard, Thanks for the KIP. I once reviewed it and was concerned about its effects on stream time advancing. After reading the updated KIP I think it has answered a lot of them already. I have a couple minor comments still, otherwise I'm +1: 1) I want to clarify that for operations result

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-02 Thread Richard Yu
Hi all, Thanks for the votes so far! @Matthias or @Guozhang Wang it would be great to also get your input on this KIP. It looks to be pretty close to completion, so the finishing touches are all we need. :) Best, Richard On Mon, Mar 2, 2020 at 11:45 AM Ghassan Yammine < ghassan.yamm...@bazaarv

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-02 Thread Ghassan Yammine
Hello all, +1 (non-binding) Thanks, Ghassan On 3/2/20, 12:43 PM, "Bruno Cadonna" wrote: EXTERNAL: This email originated from outside of Bazaarvoice. Do not click any links or open any attachments unless you trust the sender and know the content is safe. Hi Richard,

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-02 Thread Bruno Cadonna
Hi Richard, +1 (non-binding) Best, Bruno On Mon, Mar 2, 2020 at 4:33 PM John Roesler wrote: > > Hi Richard, > > Thanks for the KIP! > > I'm +1 (binding) > > -john > > On Thu, Feb 27, 2020, at 14:40, Richard Yu wrote: > > Hi all, > > > > I am proposing a new optimization to Kafka Streams which w

Re: [VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-03-02 Thread John Roesler
Hi Richard, Thanks for the KIP! I'm +1 (binding) -john On Thu, Feb 27, 2020, at 14:40, Richard Yu wrote: > Hi all, > > I am proposing a new optimization to Kafka Streams which would greatly > reduce the number of idempotent updates (or no-ops) in the Kafka Streams > DAG. > A number of users ha

[VOTE] KIP-557: Add emit on change support for Kafka Streams

2020-02-27 Thread Richard Yu
Hi all, I am proposing a new optimization to Kafka Streams which would greatly reduce the number of idempotent updates (or no-ops) in the Kafka Streams DAG. A number of users have been interested in this feature, so it would be nice to pass this one in. For information, the KIP is described below