Hello LF and Vinay, With the introduction of “allowed lateness” elements and windows are kept around until the watermark passes the window.maxTimestamp + allowed_lateness and then they are cleaned up (garbage collected)
Every element that comes in and belongs to a window that is garbage collected is dropped as super-late. Elements that are late, but no more than the allowed lateness, they are kept the window fires as before. If you know what the maximum latency is, then the best way is to set the allowed lateness to that value. Currently Flink drops super-late elements and does not provide any mechanism to manually handle these elements, BUT there are discussions about adding such a functionality that will allow you to (probably) have a separate stream with only these elements. Thanks, Kostas > On Oct 3, 2016, at 5:28 PM, lg...@yahoo.com wrote: > > Not yet. > I'm hoping a Flink export on this mailing list will reply. > > > - LF > > > > From: vinay patil <vinay18.pa...@gmail.com> > To: user@flink.apache.org > Sent: Monday, October 3, 2016 8:09 AM > Subject: Re: Regarding Late Elements > > Hi LF, > > So did you manage to get the workaround for it ? > > I am using a Custom Trigger which is similar to 1.0.3 with few changes > > Regards, > Vinay Patil > > On Mon, Oct 3, 2016 at 10:02 AM, lgfmt [via Apache Flink User Mailing List > archive.] <[hidden email] > <applewebdata://59CDAE6A-8A95-4A9B-B2E2-23053500938E>> wrote: > We have the same requirement - we cannot discard any data even if it arrives > late. > > > - LF > > > > > From: Vinay Patil <[hidden email] > <applewebdata://59CDAE6A-8A95-4A9B-B2E2-23053500938E>> > To: [hidden email] <applewebdata://59CDAE6A-8A95-4A9B-B2E2-23053500938E> > > Sent: Sunday, October 2, 2016 8:21 PM > Subject: Regarding Late Elements > > Hi Guys, > > Just wanted to get an idea on Why Flink decided to completely discard late > elements in the latest version ?, this was not the case in 1.0.3 > > > P.S In our case the data is critical so we cannot discard a single record > even if it is late, I have written a custom trigger (as suggested by > Aljoscha) to even accept late elements. > > > Regards, > Vinay Patil > > > > > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user- mailing-list-archive.2336050. > n4.nabble.com/Regarding-Late- Elements-tp9284p9292.html > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Regarding-Late-Elements-tp9284p9292.html> > To start a new topic under Apache Flink User Mailing List archive., email > [hidden email] <applewebdata://59CDAE6A-8A95-4A9B-B2E2-23053500938E> > To unsubscribe from Apache Flink User Mailing List archive., click here > <applewebdata://59CDAE6A-8A95-4A9B-B2E2-23053500938E>. > NAML > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > View this message in context: Re: Regarding Late Elements > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Regarding-Late-Elements-tp9284p9294.html> > Sent from the Apache Flink User Mailing List archive. mailing list archive > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at > Nabble.com. > > >
