Hi, there were some discussions on the ML and it seems that the consensus is to aim for a release this year.
Let me think a bit more and get back to you on the other issues. Cheers, Aljoscha On Thu, 10 Nov 2016 at 11:47 Konstantin Knauf <konstantin.kn...@tngtech.com> wrote: > Hi Aljoscha, > > unfortunately, I think, FLINK-4994 would not solve our issue. What does > "on the very end" mean in case of a GlobalWindow? > > FLINK-4369 would fix my workaround though. Is there already a timeline > for Flink 1.2? > > Cheers, > > > Konst > > On 10.11.2016 10:19, Aljoscha Krettek wrote: > > Hi Konstantin, > > evicting elements not being evicted is a bug that should be fixed for > > Flink 1.2: https://issues.apache.org/jira/browse/FLINK-4369. > > > > The check about non-existing window state when triggering was introduced > > because otherwise a Trigger could return FIRE and then there would be > > nothing to fire. I guess if we did indeed fire the trigger even with > > non-existing state then some people might wonder why no emission is > > triggered when their trigger returns FIRE. I see your point though, that > > the omitted firing is problematic for some cases. > > > > I think having clear() as proposed > > in https://issues.apache.org/jira/browse/FLINK-4994 would solve your > > case. You were using your own cleanup timer as a workaround because > > clear() is currently also called on PURGE. With clear() only being > > called at the very end this should work, correct? > > > > Cheers, > > Aljoscha > > > > On Wed, 9 Nov 2016 at 19:53 Konstantin Knauf > > <konstantin.kn...@tngtech.com <mailto:konstantin.kn...@tngtech.com>> > wrote: > > > > Hi Aljoscha, > > > > as it turns out the "workaround" I was thinking was functionally > > working, but had a so to say memory leak. I was under the impression > > that evicted elements will be removed from the window state... > > > > Anyway, I think that this (triggers not being evaluated when the > window > > state is null) turns out to be blocker for us. > > > > Why is this check done? Since a user can do basically whatever she > likes > > in onProcessingTimeTimer() the comment > > > > // if we have no state, there is nothing to do > > > > is, well, just not true in some cases (e.g. state updates in our > case). > > > > Cheers, > > > > Konstantin > > > > On 09.11.2016 14:17, Aljoscha Krettek wrote: > > > Could you go into some detail of why you need to keep the trigger > > state? > > > > > > Just the basics because you probably cannot (should not) talk > > about your > > > internal stuff. > > > > > > On Wed, 9 Nov 2016 at 13:16 Konstantin Knauf > > > <konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>>> wrote: > > > > > > Sounds good Aljoscha. > > > > > > sent from my phone. Plz excuse brevity and tpyos. > > > --- > > > Konstantin Knauf *konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>> * +49-174-3413182 > <0174%203413182> > > <tel:0174%203413182> > > > <tel:0174%203413182> > > > > > > TNG Technology Consulting GmbH, Betastr. 13a, 85774 > Unterföhring > > > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert > > Dahlke > > > > > > ---- Aljoscha Krettek schrieb ---- > > > > > > > > > Hi, > > > exactly for this case I want to make a change to when > > > Trigger.clear() is > > > called: https://issues.apache.org/jira/browse/FLINK-4994 > > > > > > Right now, clear is called when the window is being garbage > > > collected because we passed the allowed lateness (after this, > > > nothing will ever be added to a window again) and also when the > > > Trigger returns PURGE or FIRE_AND_PURGE. > > > > > > I want to change it to only be called in the former case. We > could > > > possibly add an onPurge() callback to allow cleaning state on > > purge > > > or require people to put the code that they want to run on > > PURGE in > > > the Trigger method that returns the PURGE. > > > > > > What do you think? > > > > > > Cheers, > > > Aljoscha > > > > > > On Tue, 8 Nov 2016 at 18:46 Konstantin Knauf > > > <konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>>> > > > wrote: > > > > > > Hi Aljoscha, > > > > > > interesting, this explains it. Well, in our case the PURGE > > in the > > > onProcessingTimeTimer is only used to clear > > KeyValueStates*, and > > > at this > > > point there are usually no records in the window state. > > > > > > Any Ideas? > > > > > > I do have a workaround with an evictor, but it seemed to be > > > unnecessarily complicated. > > > > > > *We can not use clear()-callback for that, since this > > state should > > > survive the FIRE_AND_PURGEs in the onElement()-calls. > > > > > > Cheers, > > > > > > Konstantin > > > > > > > > > On 08.11.2016 18:31, Aljoscha Krettek wrote: > > > > Hi, > > > > the timers are not actually deleted but the > WindowOperator > > > will check > > > > whether there is any window state associated with the > window > > > for which > > > > the timer fires. If there is no window state the timer > will > > > silently be > > > > ignored. > > > > > > > > Is this a problem for you or did you just want to > > clarify? If > > > yes, then > > > > we should work on finding a solution. > > > > > > > > Cheers, > > > > Aljoscha > > > > > > > > On Tue, 8 Nov 2016 at 18:18 Konstantin Knauf > > > > <konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>>>> wrote: > > > > > > > > Hi everyone, > > > > > > > > I just migrated a streaming Job from 1.0.2 to 1.1.3 > and > > > stumbled across > > > > a problem concerning one of our custom triggers. > > > > > > > > The trigger basically FIRE_AND_PURGEs multiple times > in > > > onElement() and > > > > the window is PURGEd onProcessingTimeTimer(), but it > > seems > > > that the all > > > > registered processing time timers are deleted > everytime > > > the window is > > > > PURGEd. > > > > > > > > clear() is the default implementation, i.e. no-op. > > > > > > > > Just wanted to, if this is the expected behavior > > > (processing time timers > > > > being deleted on PURGE or FIRE_AND_PURGE) from Flink > > 1.1 on? > > > > > > > > Cheers, > > > > > > > > Konstantin > > > > > > > > -- > > > > Konstantin Knauf * konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>> > > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>>> * +49-174-3413182 > <0174%203413182> > > <tel:0174%203413182> > > > <tel:0174%203413182> > > > > <tel:0174%203413182> > > > > TNG Technology Consulting GmbH, Betastr. 13a, 85774 > > > Unterföhring > > > > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. > > > Robert Dahlke > > > > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > > > > > > > > > > > > > > -- > > > Konstantin Knauf * konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> > > > <mailto:konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com>> * +49-174-3413182 > <0174%203413182> > > <tel:0174%203413182> > > > <tel:0174%203413182> > > > TNG Technology Consulting GmbH, Betastr. 13a, 85774 > > Unterföhring > > > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. > > Robert Dahlke > > > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > > > > > > > -- > > Konstantin Knauf * konstantin.kn...@tngtech.com > > <mailto:konstantin.kn...@tngtech.com> * +49-174-3413182 > <0174%203413182> > > <tel:0174%203413182> > > TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring > > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke > > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > > > > -- > Konstantin Knauf * konstantin.kn...@tngtech.com * +49-174-3413182 > <0174%203413182> > TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > >
