Thanks for the proposal Jingguo. Guozhang when you mention to "change the default value of retention to 0 consistently", will we introduce any backward incompatible issue since the retention cutoff might change on the changelog topics that are unexpected.
Boyang ________________________________ From: Guozhang Wang <wangg...@gmail.com> Sent: Friday, January 4, 2019 3:16 AM To: dev Subject: Re: Suggestion to make 0 grace period the default for suppress Thanks for reporting this Jingguo, personally I'd favor we change the default value of retention to 0 consistently, regardless of whether suppress is used, if we would ever consider changing it. Since it is a public API change (i.e. changing the default value), it should be discussed and voted via a KIP: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals Are you interested in creating one and drive the discussion? Guozhang On Sun, Dec 30, 2018 at 10:34 PM jingguo yao <yaojing...@gmail.com> wrote: > [1] has the following code to demonstrate the usage of suppress method. > > KGroupedStream<UserId, Event> grouped = ...; > grouped > .windowedBy(TimeWindows.of(Duration.ofHours(1)).grace(ofMinutes(10))) > .count() > .suppress(Suppressed.untilWindowCloses(unbounded())) > .filter((windowedUserId, count) -> count < 3) > .toStream() > .foreach((windowedUserId, count) -> > sendAlert(windowedUserId.window(), windowedUserId.key(), count)); > > > If I remove the grace method invocation, I will have a one day > retention period. The following code in > org.apache.kafka.streams.kstream.TimeWindows causes this hehaviour: > > @SuppressWarnings("deprecation") // continuing to support > Windows#maintainMs/segmentInterval in fallback mode > @Override > public long gracePeriodMs() { > // NOTE: in the future, when we remove maintainMs, > // we should default the grace period to 24h to maintain the default > behavior, > // or we can default to (24h - size) if you want to be super accurate. > return grace != null ? grace.toMillis() : maintainMs() - size(); > } > > I think that it is better to use 0 grace period if > "suppress(Suppressed.untilWindowCloses(unbounded()))" exists. With the > suppress method invocation, people are expecting to see the final > window result when the window closes instead of wait to see the result > after the one-day period. Even if we have some reasons similar to ones > mentioned in the code comment, it is better to mention this hehaviour > somewhere in Kafka streams documentation. > > > [1] > http://kafka.apache.org/21/documentation/streams/developer-guide/dsl-api.html#window-final-resu > > > -- > Jingguo > -- -- Guozhang
