Hi, I've just elevated FLINK-5479 to BLOCKER for 1.5.
Unfortunately, AFAIK there is no easy workaround solution for this issue yet in the releases so far. The min watermark logic that controls per-partition watermark emission is hidden inside the consumer, making it hard to work around it. One possible solution I can imagine, but perhaps not that trivial, is to inject some special marker event into all partitions periodically. The watermark assigner should be able to recognize this special marker and try to provide some watermark for it. Another option is that I can provide some patch you can apply for a custom build of the Kafka connector that handles partition idleness properly. However, given that we're aiming for a faster release cycle for Flink 1.5 (proposed release date is Feb. 2018), it might not be worth the extra maintenance effort on your side of a custom build. Best, Gordon On Tue, Dec 12, 2017 at 9:28 PM, gerardg <ger...@talaia.io> wrote: > I'm also affected by this behavior. There are no updates in FLINK-5479 but > did you manage to find a way to workaround this? > > Thanks, > > Gerard > > > > -- > Sent from: http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble.com/ >
