Hi Sujit, this does indeed sound strange and we are not aware of any data loss issues. Are there any exceptions or other errors in the job/taskmanager logs? Do you have a minimal working example? Is it that whole windows are not processed or just single items inside a window?
Nico On Tuesday, 14 February 2017 16:57:31 CET Sujit Sakre wrote: > Hi, > > I have been using Flink 1.1.1 with Kafka 0.9 to process real time streams. > We have written our own Window Function and are processing data with > Sliding Windows. We are using Event Time and use a custom watermark > generator. > > We select a particular window out of multiple sliding windows and process > all items in the window in an iterative loop to increment counts of the > items selected. After this we call the sink method to log the result in a > database. > > This is working fine most of the times, i.e. it produces the expected > result most of the times. However there are situations when certain windows > are not processed (under same test conditions), this means the results are > less than expected i.e. the counts are less than expected. Sometimes, items > instead of certain windows not getting processed, certain items do not get > processed in the window. This is unpredictable. > > I wish to know what could be the cause of this inconsistent behaviour. How > do we resolve it. We have integrated 1.2 and Kafka 0.10 now, the problem > persists. > > Please could you suggest about what the problem could be and how to resolve > this. > > Many thanks. > > > *Sujit Sakre*
signature.asc
Description: This is a digitally signed message part.
