Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-12-01 Thread Matthias J. Sax
Yes! Sorry for that typo. On 12/1/16 5:07 PM, Guozhang Wang wrote: > You mean "it is a backward incompatible change" right? > > On Wed, Nov 30, 2016 at 4:28 PM, Matthias J. Sax > wrote: > >> Thanks for this clarification (and your +1) >> >> I completely agree and just want to add my thoughts: >

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-12-01 Thread Guozhang Wang
You mean "it is a backward incompatible change" right? On Wed, Nov 30, 2016 at 4:28 PM, Matthias J. Sax wrote: > Thanks for this clarification (and your +1) > > I completely agree and just want to add my thoughts: > > 1. Yes, it is a backward compatible change but as I discusses with > others, w

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-30 Thread Matthias J. Sax
Thanks for this clarification (and your +1) I completely agree and just want to add my thoughts: 1. Yes, it is a backward compatible change but as I discusses with others, we want accept this for now. All previous releases did contain non-compatible changes for Kafka Streams, too. And as Kafka St

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-29 Thread Ewen Cheslack-Postava
I think this looks reasonable, but just a more general note on compatibility -- I think it's worth trying to clarify what types of compatibility we're trying to achieve. Guozhang's 1 & 2 give an important breakdown (compile vs runtime compatibility). For the type of change described here, I think i

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-28 Thread Matthias J. Sax
Done. If there is no further comments, I would like to start a voting thread in a separate email. -Matthias On 11/28/16 9:08 AM, Guozhang Wang wrote: > Yes it does not include these, again in my previous previous email I meant > when you say "This is a breaking, incompatible change" people may i

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-28 Thread Guozhang Wang
Yes it does not include these, again in my previous previous email I meant when you say "This is a breaking, incompatible change" people may interpret it differently. So better explain it more clearly. Guozhang On Thu, Nov 24, 2016 at 10:31 PM, Matthias J. Sax wrote: > That does make sense. Bu

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-24 Thread Matthias J. Sax
That does make sense. But KIP-93 does not change anything like this, so there is nothing to mention, IMHO. Or do you mean, the KIP should include that the change is backward compatible with this regard? -Matthias On 11/24/16 5:31 PM, Guozhang Wang wrote: > What I meant is that, for some change

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-24 Thread Guozhang Wang
What I meant is that, for some changes (e.g. say we change the auto-repartition behavior that caused using different name conventions, or some changes that involve changing the underlying state store names, etc) the existing internal state including the stores and topics will probably not valid. So

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-23 Thread Matthias J. Sax
Thanks for the feedback. I updated the KIP for (1) and (2). However not for (3): Why should it be required to reset an application? If user processed "good" data with valid timestamps, behavior does not change. If user tried to process "bad" data with invalid timestamps, the application does fail

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-22 Thread Guozhang Wang
Regarding the "compatibility" section, I would suggest being a bit more specific about why it is a breaking change. For Streams, it could mean different things: 1. User need code change when switching library dependency on the new version, otherwise it won't compile(I think this is the case for th

[DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

2016-11-18 Thread Matthias J. Sax
Hi all, I want to start a discussion about KIP-93: https://cwiki.apache.org/confluence/display/KAFKA/KIP-93%3A+Improve+invalid+timestamp+handling+in+Kafka+Streams Looking forward to your feedback. -Matthias signature.asc Description: OpenPGP digital signature