This is possibly related to the way the queue between StreamIterationTail and Head is currently implemented. I think this part is a bit prone to records loss when things get wacky and backpressure kicks in (but at least it avoids deadlocks, right?).
I don’t have the time availability to look into the code right now but I am going to focus on a progress/loops by September, hopefully with a FLIP, which solves that part as well. If this is urgent, please go ahead and check this now, I think that queue timeouts cause this... Paris PS: on my yet incomplete PR (I know I know) I basically disabled queue polling timeouts since the checkpoint overhead on the StreamIterationHead almost always led to record loss. https://github.com/apache/flink/pull/1668 On 20 Jul 2016, at 11:57, Maximilian Michels <m...@apache.org<mailto:m...@apache.org>> wrote: CC Gyula and Paris in case they might want to help out. On Tue, Jul 19, 2016 at 11:43 AM, Biplob Biswas <revolutioni...@gmail.com<mailto:revolutioni...@gmail.com>> wrote: Hi Ufuk, Thanks for the update, is there any known way to fix this issue? Any workaround that you know of, which I can try? -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-point-goes-missing-within-iteration-tp7776p8015.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com<http://nabble.com>.
