Re: Apache Flink - Clarifications about late side output

2019-12-11 Thread M Singh
Thanks David for your detailed answers.   Mans On Wednesday, December 11, 2019, 08:12:51 AM EST, David Anderson wrote: If we have allowed lateness to be greater than 0 (say 5), then if an event which arrives at window end + 3 (within allowed lateness),      (a) it is considered late

Re: Apache Flink - Clarifications about late side output

2019-12-11 Thread David Anderson
> > If we have allowed lateness to be greater than 0 (say 5), then if an event > which arrives at window end + 3 (within allowed lateness), > > (a) it is considered late and included in the window function as a > late firing ? > An event with a timestamp that falls within the window's boundarie

Re: Apache Flink - Clarifications about late side output

2019-12-11 Thread David Anderson
I'll attempt to answer your questions. If we have allowed lateness to be greater than 0 (say 5), then if an event > which arrives at window end + 3 (within allowed lateness), > (a) it is considered late and included in the window function as a > late firing ? > An event with a timestamp that

Re: Apache Flink - Clarifications about late side output

2019-12-11 Thread M Singh
Thanks Timo for your answer.  I will try the prototype but was wondering if I can find some theoretical documentation to give me a sound understanding. Mans On Wednesday, December 11, 2019, 05:44:15 AM EST, Timo Walther wrote: Little mistake: The key must be any constant instead of `e`

Re: Apache Flink - Clarifications about late side output

2019-12-11 Thread Timo Walther
Little mistake: The key must be any constant instead of `e`. On 11.12.19 11:42, Timo Walther wrote: Hi Mans, I would recommend to create a little prototype to answer most of your questions in action. You can simple do: stream = env.fromElements(1L, 2L, 3L, 4L)    .assignTimestampsAndWater

Re: Apache Flink - Clarifications about late side output

2019-12-11 Thread Timo Walther
Hi Mans, I would recommend to create a little prototype to answer most of your questions in action. You can simple do: stream = env.fromElements(1L, 2L, 3L, 4L) .assignTimestampsAndWatermarks( new AssignerWithPunctuatedWatermarks{ extractTimestamp(e) = e, check

Apache Flink - Clarifications about late side output

2019-12-10 Thread M Singh
Hi: I have a few questions about the side output late data.   Here is the API stream .keyBy(...) <- keyed versus non-keyed windows .window(...) <- required: "assigner" [.trigger(...)]<- optional: "trigger" (else default trigger) [.