Hi Ewen, Personally I'm thinking to remove deprecated APIs after one minor release, so yes, 0.12.0.0 would be the preferred timeline.
Guozhang On Thu, Apr 6, 2017 at 8:40 PM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > I agree there's a potential issue that Eno raises, but probably it's not > worth worrying about trying to enforce it. It seems like we have the right > default for long term and for most cases a custom timestamp extractor > should only be needed for legacy data that isn't putting a timestamp in the > record. I think that if the user chooses to customize the timestamp > extractor, it's reasonable to expect them to read enough docs to understand > the implications. (By the way, this is also, I think a case where fewer > overloads are fine because it should be unusual and a relatively advanced > use case, so expecting the user to fill in some extra parameters even if > they are using the default values is ok in order to keep the API tighter. > However, if we're heading towards a builder API that won't matter anyway.) > > One other question I have is what the planned removal timeline is for the > deprecated configs -- the KIP just says they will be deprecated but not > when they will be removed. Specifying this is important for users so they > can plan how aggressively they need to switch. Generally removal is tied to > major releases, so I assume the thought is to remove in 0.12.0.0? > > -Ewen > > On Tue, Feb 28, 2017 at 9:21 AM, Matthias J. Sax <matth...@confluent.io> > wrote: > > > Eno, > > > > I think this problem is out-of-scope and also present in the current > > setting. We cannot avoid that a custom timestamp extractor uses and > > if-else branch and returns different timestamps for different topic. > > That is possible even right now. > > > > Furthermore, the TimestampExtractor interface states: > > > > > The extracted timestamp MUST represent the milliseconds since midnight, > > January 1, 1970 UTC. > > > > If uses don't follow this, there is nothing we can do about it. > > > > > > -Matthias > > > > > > On 2/28/17 7:47 AM, Jeyhun Karimov wrote: > > > Hi Eno, > > > > > > Thanks for clarification. I think it is by definition allowed. So if > we > > > want to join a stream that uses wallclock time with a stream that uses > an > > > event time, then we can assign the first one a timestamp extractor that > > > returns system clock, and for the second stream we can assign timestamp > > > extractor that extracts/computes the event time from record. > > > > > > Cheers, > > > Jeyhun > > > > > > On Tue, Feb 28, 2017 at 11:40 AM Eno Thereska <eno.there...@gmail.com> > > > wrote: > > > > > >> Hi Jeyhun, > > >> > > >> I mean something slightly different. In your motivation you say > "joining > > >> multiple streams/tables that require different timestamp extraction > > >> methods". I wan to understand the scope of this. Is it allowed to > have a > > >> stream that uses wallclock time join a stream that uses event time? > (It > > >> would be good to give some examples in the motivation about scenarios > > you > > >> envision). If the join is not allowed, how do you prevent that join > from > > >> happening? Do you throw an exception? > > >> > > >> Thanks > > >> Eno > > >> > > >> > > >>> On 28 Feb 2017, at 10:04, Jeyhun Karimov <je.kari...@gmail.com> > wrote: > > >>> > > >>> Hi Eno, > > >>> > > >>> Thanks for feedback. I think you mean [1]. In this KIP we do not > > consider > > >>> the situations you mentioned. So, either we can extend the KIP and > > solve > > >>> mentioned issues or submit 2 PRs incrementally. > > >>> > > >>> [1] https://issues.apache.org/jira/browse/KAFKA-4785 > > >>> > > >>> > > >>> Cheers, > > >>> Jeyhun > > >>> > > >>> On Tue, Feb 28, 2017 at 10:41 AM Eno Thereska < > eno.there...@gmail.com> > > >>> wrote: > > >>> > > >>>> Hi Jeyhun, > > >>>> > > >>>> Thanks for the KIP, sorry I'm coming a bit late to the discussion. > > >>>> > > >>>> One thing I'd like to understand is whether we can avoid situations > > >> where > > >>>> the user is mixing different times (event time vs. wallclock time) > in > > >> their > > >>>> processing inadvertently. Before this KIP, all the relevant topics > > have > > >> one > > >>>> time stamp extractor so that issue does not come up. > > >>>> > > >>>> What will be the behavior if times mismatch, e.g., for joins? > > >>>> > > >>>> Thanks > > >>>> Eno > > >>>> > > >>>>> On 22 Feb 2017, at 09:21, Jeyhun Karimov <je.kari...@gmail.com> > > wrote: > > >>>>> > > >>>>> Dear community, > > >>>>> > > >>>>> I would like to get further feedbacks on this KIP (if any). > > >>>>> > > >>>>> Cheers > > >>>>> Jeyhun > > >>>>> > > >>>>> On Wed, Feb 15, 2017 at 2:36 AM Matthias J. Sax < > > matth...@confluent.io > > >>> > > >>>>> wrote: > > >>>>> > > >>>>>> Mathieu, > > >>>>>> > > >>>>>> I personally agree with your observation, and we have plans to > > submit > > >> a > > >>>>>> KIP like this. If you want to drive this discussion feel free to > > start > > >>>>>> the KIP by yourself! > > >>>>>> > > >>>>>> Having said that, for this KIP we might want to focus the > discussion > > >> the > > >>>>>> the actual feature that gets added: allowing to specify different > > >>>>>> TS-Extractor for different inputs. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> -Matthias > > >>>>>> > > >>>>>> On 2/14/17 4:54 PM, Mathieu Fenniak wrote: > > >>>>>>> Hi Jeyhun, > > >>>>>>> > > >>>>>>> This KIP might not be the appropriate time, but my first thought > > >>>> reading > > >>>>>> it > > >>>>>>> is that it might make sense to introduce a builder-style API > rather > > >>>> than > > >>>>>>> adding a mix of new method overloads with independent optional > > >>>>>> parameters. > > >>>>>>> :-) > > >>>>>>> > > >>>>>>> eg. stream(), table(), globalTable(), addSource(), could all > > accept a > > >>>>>>> "TopicReference" parameter that can be built like: > > >>>>>>> > > >>>>>> > > >>>> > > >> TopicReference("my-topic").keySerde(...).valueSerde(...). > > autoOffsetReset(...).timestampExtractor(...).build(). > > >>>>>>> > > >>>>>>> Mathieu > > >>>>>>> > > >>>>>>> > > >>>>>>> On Tue, Feb 14, 2017 at 5:31 PM, Jeyhun Karimov < > > >> je.kari...@gmail.com> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Dear community, > > >>>>>>>> > > >>>>>>>> I want to share the KIP-123 [1] which is based on issue > KAFKA-4144 > > >>>> [2]. > > >>>>>> You > > >>>>>>>> can check the PR in [3]. > > >>>>>>>> > > >>>>>>>> I would like to get your comments. > > >>>>>>>> > > >>>>>>>> [1] > > >>>>>>>> > > >>>>>> > > >>>> > > >> https://cwiki.apache.org/confluence/pages/viewpage. > > action?pageId=68714788 > > >>>>>>>> [2] https://issues.apache.org/jira/browse/KAFKA-4144 > > >>>>>>>> [3] https://github.com/apache/kafka/pull/2466 > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> Cheers, > > >>>>>>>> Jeyhun > > >>>>>>>> -- > > >>>>>>>> -Cheers > > >>>>>>>> > > >>>>>>>> Jeyhun > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>>> -- > > >>>>> -Cheers > > >>>>> > > >>>>> Jeyhun > > >>>> > > >>>> -- > > >>> -Cheers > > >>> > > >>> Jeyhun > > >> > > >> -- > > > -Cheers > > > > > > Jeyhun > > > > > > > > -- -- Guozhang
