Thanks Fabian and Stefan for all the help.
Best Regards,
> On 22 Jun 2017, at 18:06, Fabian Hueske wrote:
>
> 1].
You have to register an event-time timer in the `processElement()` method.
You'll get a callback to `onTimer()` when the function receives a watermark
that is greater than the registered timer. So you can always register a
timer for the end time of the next window to get a call back to `onTimer()`
Hi Fabian,
How the process function will be called at 12:00:01 as there are no windows
output or events after 12:00:00.
Thanks
> On 22 Jun 2017, at 17:07, Fabian Hueske wrote:
>
> Let's say window A and window B end at 12:00:00 and window C at 13:00:00.
> When the ProcessFunction receives a
Let's say window A and window B end at 12:00:00 and window C at 13:00:00.
When the ProcessFunction receives a watermark at 12:00:01, it knows that
Window A and B have been finished.
When it receives a watermark of 13:00:01 it knows that all results of
window C have been received. If there were no r
Thanks for the answers. My scenario is:
| Window A |
| Window B |
| Window C |
If no events are received for Window C, then how process function would
know that both window 'A' and window 'B' have finished and need to
aggregated their result before sink is called?
Thanks
On
Hi Ahmad,
Flink's watermark mechanism guarantees that when you receive a watermark
for time t all records with a timestamp smaller than t have been received
as well.
Records emitted from a window have the timestamp of their end time. So, the
ProcessFunction receives a timestamp for 12:00:00 you ca
Hi Stefan,
How process function would know that the last window result has arrived?
Because slidingwindows slide every 5 minutes which means that window of new
time-range or new watermark will arrive after 5 minutes.
Thanks
On 22 June 2017 at 15:10, Stefan Richter
wrote:
> The process functio
The process function has the signature
void processElement(I value, Context ctx, Collector out) throws Exception
where the context is providing access to the current watermark and you can also
register timer callbacks, when that trigger when a certain watermark is
reached. You can simply monitor
Thanks Stefan, But how the Process function will have these watermarks? I
have sliding windows like below
final DataStream eventStream = inputStream
.keyBy(TENANT, CATEGORY)
.window(SlidingProcessingTimeWindows.of(Time.minutes(100,Time.minutes(5)))
.fold(new WindowStats(), new ProductAggregatio
Hi,
one possible approach could be that you have a process function before the
sink. Process function is aware of watermarks, so it can collect and buffer
window results until it sees a watermark. This is the signal that all results
for windows with an end time smaller than the watermark are co
Hi All,
I am using categoryID as a keyby attribute for creating keyed stream from a
product event stream. Keyed stream then creates time windows for each
category. However, when the window time expires, i want to write the output
data of all the products in all all categories in a single atomic op
11 matches
Mail list logo
