Re: Flink custom trigger use case

2021-02-25 Thread Roman Khachatryan
Hi, Yes, you have an Iterable with window elements as the ProcessWindowFunction input. You can then emit them individually. Regards, Roman On Thu, Feb 25, 2021 at 7:22 AM Diwakar Jha wrote: > Hello, > > I tried using *processWindowFunction* since it gives access to > *globalstate* through *co

Re: Flink custom trigger use case

2021-02-24 Thread Diwakar Jha
Hello, I tried using *processWindowFunction* since it gives access to *globalstate* through *context*. My question is, Is it possible to discard single events inside *process* function of *processWindowFunction* just like *onElements* of triggers? For my use case it seems that trigger is not suffi

Re: Flink custom trigger use case

2021-02-24 Thread Diwakar Jha
Hi Arvid, Thanks. I tried FIRE instead of FIRE_AND_PURGE and it introduced duplicates though the result is still the same i.e record 1 is fired both at the start and the end of the window. so for every window i see the first event of the window is coming twice in the output. I'm trying to explain

Re: Flink custom trigger use case

2021-02-24 Thread Arvid Heise
Hi Diwakar, the issue is that you fire_and_purge the state, you should just FIRE on the first element (or else you lose the information that you received the element already). You'd use FIRE_AND_PURGE on the last element though. On Wed, Feb 24, 2021 at 7:16 AM Khachatryan Roman < khachatryan.ro..

Re: Flink custom trigger use case

2021-02-23 Thread Khachatryan Roman
Hi Diwakar, I'm not sure I fully understand your question. If event handling in one window depends on some other windows than TriggerContext.getPartitionedState can not be used. Triggers don't have access to the global state (only to key-window scoped state). If that's what you want then please co

Re: Flink custom trigger use case

2021-02-23 Thread Diwakar Jha
Hi Roman, Thanks for your reply! That was a typo, i'm using TumblingProcessingTimeWindows My problem is that i want to stop the first event trigger (per key) except for the first window. right now, my first event is getting triggered in every window. Will setting "*state (firstSeen) value is true

Re: Flink custom trigger use case

2021-02-23 Thread Roman Khachatryan
Hi, I've noticed that you are using an event time window, but the trigger fires based on processing time. You should also register an event time timer (for the window end). So that trigger.onEventTime() will be called. And it's safer to check if the state (firstSeen) value is true, not just exists

Fwd: Flink custom trigger use case

2021-02-23 Thread Diwakar Jha
Hello, posting again for help. I'm planning to use state TTL but would like to know if there is any other way to do it. I'm using Flink 1.11. Thanks! -- Forwarded message - From: Diwakar Jha Date: Mon, Feb 22, 2021 at 6:28 PM Subject: Flink custom trigger use cas

Flink custom trigger use case

2021-02-22 Thread Diwakar Jha
Hello, I'm trying to use a custom trigger for one of my use case. I have a basic logic (as shown below) of using keyBy on the input stream and using a window of 1 min. .keyBy() .window(TumblingEventTimeWindows.of(Time.seconds(60))) .trigger(new CustomTrigger()) .aggregate(Input.getAggregationFunc