Re: Global Window, Trigger and Watermarks, Parallelism

2018-03-14 Thread dim5b
By adding , AfterMatchSkipStrategy.skipPastLastEvent() it returns what i want. Is there a way to track/emit "ongoing" events i.e before the pattern matchs the end event type? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Global Window, Trigger and Watermarks, Parallelism

2018-03-14 Thread dim5b
I see you replied on https://stackoverflow.com/questions/48028061/flink-cep-greedy-matching with a known bug issue on the https://issues.apache.org/jira/browse/FLINK-8914 In my case my pattern looks like Pattern tripPattern = Pattern.begin("start").times(1).where(START_CONDI

Re: Global Window, Trigger and Watermarks, Parallelism

2018-03-13 Thread dim5b
I have looked into the CEP library. I have posted an issued on stackoverflow. https://stackoverflow.com/questions/49047879/global-windows-in-flink-using-custom-triggers-vs-flink-cep-pattern-api However the pattern matches all possible solution on the stream of events.Does pattern have a notion o

Re: Global Window, Trigger and Watermarks, Parallelism

2018-03-13 Thread Chesnay Schepler
Hello, Event-time and watermarks can be used to deal with out-of-order events, but since you're using global windows (opposed to time-based windows) you have to implement the logic for doing this yourself. Conceptually, what you would have to do is to not create your TripEv when receiving a

Re: global window trigger

2017-07-20 Thread Aljoscha Krettek
Hi, Yes, you can have state in a WindowFunction if you use Flink’s state abstraction that you can access from a RichWindowFunction using the RuntimeContext. (Or by using a ProcessWindowFunction). Trigger purging behaviour makes a difference if the Trigger fires repeatedly before the watermark

Re: global window trigger

2017-07-20 Thread Aljoscha Krettek
Hi, I’m afraid this will not work well because a WindowAssigner should be stateless, i.e. it should not keep any state in fields. The reason is that there can be several WindowAssigners used on the different partitions and the order in which a WindowAssigner sees the incoming elements is also n

Re: global window trigger

2017-07-16 Thread Aljoscha Krettek
Hi, Ok, then I misunderstood. Yes, a PurgingTrigger it similar (the same) to always returning FIRE_AND_PURGE instead of FIRE in a custom Trigger. I thought your problem was that data is never cleared away when using GlobalWindows. Is that not the case? Best, Aljoscha > On 14. Jul 2017, at 16:2

Re: global window trigger

2017-07-14 Thread jad mad
Hi Aljoscha thanks for the comment. is wrapping by a PurgingTrigger.of() the same as doing "return TriggerResult.FIRE_AND_PURGE;" inside of a custom trigger? gave it a test and the result seems the opposite of what I meant... instead of throwing away previous windows' contents, I wanna keep them

Re: global window trigger

2017-07-13 Thread Aljoscha Krettek
Window contents are only purged from state if the Trigger says so or if the watermark passes the garbage collection horizon for a given window. With GlobalWindows, the GC horizon is never reached, that leaves Triggers. You can create a Trigger that purges every time it fires by wrapping it in a

Re: global window trigger

2017-07-13 Thread jad mad
Hi Prashant, env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime); actually I could make my custom trigger to fire periodically. The problem is the element set stored in the iterable variable is always uniform which is not what I'm expecting... private static class MyWindowFunction_Wind

Re: global window trigger

2017-07-12 Thread prashantnayak
Hi We've have custom operators using global windows and are using event time. How are you specifying event time as the time characteristic? Prashant -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/global-window-trigger-tp14206p14239.html