Thanks Leah! This kind of assumes an implicit answer to Matthias's question, but I was wondering if we should take this opportunity to choose a better default value for the grace period. Note that the default of -1 in the TimeWindows class, for example, ultimately gets translated into a default value of 24 hours. Which is kind of a long time.
This has hit users of suppression especially hard, since it means you won't see *any* output for 24 hours. It's pretty frustrating when you're trying to unit test your topology and nothing happens just because you didn't explicitly override the default grace period. Unfortunately, we're stuck with the 24hr grace period for our existing operators for compatibility reasons. But since this is a new kind of aggregation, we have the opportunity to consider alternatives and try to improve on this pain point for users. Of course, the obvious question now is: what would be a good grace period? We've discussed this a number of times before and as far as I know were never came up with a good answer. It might also be somewhat confusing for users if different kinds of windowed aggregations had different default grace periods, although that's not a great reason to keep doing something that's obviously causing problems. On the other hand, someone recently brought up what I thought was a good suggestion: just make the grace period a required parameter. This seems to solve the existing problem while dodging the question of what a "good" universal default would be. WDYT? Cheers, Sophie On Thu, Jul 9, 2020 at 9:47 PM Matthias J. Sax <mj...@apache.org> wrote: > Leah, > > thanks a lot for the KIP. Very well written. > > The KIP does not talk about the handling of out-of-order data though. > How do you propose to address this? > > > -Matthias > > On 7/8/20 5:33 PM, Leah Thomas wrote: > > Hi all, > > I'd like to kick-off the discussion for KIP-450, adding sliding window > > aggregation support to Kafka Streams. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-450%3A+Sliding+Window+Aggregations+in+the+DSL > > > > Let me know what you think, > > Leah > > > >
