Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Kostas Kloudas
Thanks Aljioscha In fact the onFire() method proposed in the JIRA is also included in the FLIP. The onCleanup() I agree that it would be a nice addition as it makes the API more complete. Now we have an onMerge(), an onFire() and onCleanup() which allow a trigger to react to every milestone in

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Aljoscha Krettek
Hi, I opened this Jira which should help in implementing the Trigger DSL but is also independent in that it just enhances the range of things that can be done with a Trigger: https://issues.apache.org/jira/browse/FLINK-4415 Cheers, Aljoscha On Wed, 17 Aug 2016 at 14:38 Jark Wu wrote: > Hi Aljos

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Jark Wu
Hi Aljoscha, Kostas, thanks for your detailed explanation. It makes sense. According to the discarding and accumulating, the FLIP says “the mode of parent trigger overwrites that of its children”. That means Trigger decide whether to discard window contents after firing , right ? But I find the

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Aljoscha Krettek
Hi, I think that would blow up state since there can be several triggers that need this kind of state, Any and All come to mind, possibly. If each of those keeps state that's at least a byte per trigger. If the finished state were kept centrally by the TriggerRunner it would just be one byte for ev

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Kostas Kloudas
Hi Aljoscha, On the Repeat.? addition, I think that each trigger will have to have its own implementation, e.g. the CountTrigger should just set a dummy value in the counter in order to know if it should fire again or not. In other case, we will have to add more state and this can lead to signifi

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Aljoscha Krettek
Regarding Repeat.forever() and the default being to not repeat. The simple reason is that Beam (née Google Dataflow) provides basically the same thing with their trigger DSL and that their triggers behave like this. I think it would not be beneficial to have the same feature in two systems in that

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Aljoscha Krettek
Kostas already nicely explained this! I just want to give some theoretical background. I see the underlying idea of triggers similar to predicates, i.e. "EventTimeTrigger.afterEndOfWindow().withEarlyTrigger(earlyFiringTrigger)" translates to a predicate "(E and ET) or WT" (where E is a predicate t

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-17 Thread Kostas Kloudas
Hello Jark Wu, Both of them will work in the new DSL. The idea is that there should be no restrictions on the combinations one can do. Coming to what does the early and the late trigger do, the early trigger will be responsible for specifying when the trigger should fire in the period between

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Jark Wu
Hi, It’s a cool design, I really like it ! I have two questions here. The first is whether do we have the complex composite triggers, i.e. nested All and Any. Such as : Any( All(trigger1, trigger2), Any(trigger3, trigger4) ) Can the above code work? Another question is : In composit

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Kostas Kloudas
Hi Aljoscha, Thanks for the feedback! It is a nice feature to have. The reason it is not included in the FLIP is that I have not seen somebody asking for something similar in the mailing list. A point that I have to add is that it seems (from the user ML) that most of the times users expect t

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Aljoscha Krettek
Ah, I just read the document again and noticed that it might be good to differentiate between repeatable triggers and non-repeating triggers. I'm proposing to make most triggers non-repeating with the addition of a trigger that makes other triggers repeatable. Example Non-Repeating: EventTimeTrigg

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Kostas Kloudas
Thanks Till! Kostas > On Aug 16, 2016, at 3:30 PM, Till Rohrmann wrote: > > Cool design doc Klou. It's well described with a lot of details. I like it > a lot :-) +1 for implementing the trigger DSL. > > Cheers, > Till > > On Tue, Aug 16, 2016 at 3:18 PM, Kostas Kloudas > wrote: > >> Thanks

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Till Rohrmann
Cool design doc Klou. It's well described with a lot of details. I like it a lot :-) +1 for implementing the trigger DSL. Cheers, Till On Tue, Aug 16, 2016 at 3:18 PM, Kostas Kloudas wrote: > Thanks for the feedback Ufuk! > I will do that. > > > On Aug 16, 2016, at 1:41 PM, Ufuk Celebi wrote:

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Kostas Kloudas
Thanks for the feedback Ufuk! I will do that. > On Aug 16, 2016, at 1:41 PM, Ufuk Celebi wrote: > > Hey Kostas! Thanks for sharing the documents. I think it makes sense > to merge the two documents by moving the Google doc contents to the > Wiki. I think they form one unit. > > On Tue, Aug 16,

Re: [DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Ufuk Celebi
Hey Kostas! Thanks for sharing the documents. I think it makes sense to merge the two documents by moving the Google doc contents to the Wiki. I think they form one unit. On Tue, Aug 16, 2016 at 12:34 PM, Kostas Kloudas wrote: > Hi all! > > I've created a FLIP for the trigger DSL. This is the tri

[DISCUSS] FLIP-9: Trigger DSL

2016-08-16 Thread Kostas Kloudas
Hi all! I've created a FLIP for the trigger DSL. This is the triggers that we want Apache Flink to support out-of-the-box. This proposal builds on various discussions on the mailing list and aims at serving as a base for further ones. https://cwiki.apache.org/confluence/display/FLINK/FLIP-9%3A+T