Allowed lateness for coGroup/join operators

Hi folks I'm surprised coGroup and join doesn't support allowedLateness feature (while underlying WindowedStream does) in the current release (1.5/1.6). I've opened a jira ticket with more details and ready to make appropriate changes in API: https://issues.apache.org/jira/browse/FLINK-10050 I'm

Kinesis connector shard discovery mechanism

Hi there Flink Kinesis consumer checks shards id for a particular pattern: "^shardId-\\d{12}" https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/model/StreamShardHandle.java#L132 While this inlines with

Watermark alignment during unit tests

Hi devs During the work on https://issues.apache.org/jira/browse/FLINK-10050 I've found unstable behaviour of unit tests for unioned streams (which are used in CoGroupedStream/JoinedStream under the hood). Let's assume we have late elements in one of the stream. The thing is we have no guarantees

Sort streams in windows

Hi folks I want to sort stream based on event time field derived from events. To do this I can use one of the existing windows like TimeWindow to collect events in a window of a particular size, or SlidingWindow to run sort logic more often (and sort within slide). Ideally, I want to sort events a

Re: Sort streams in windows

like "sorted map state" will probably > be needed in order to efficiently query the state for minimal timestamp. > A less efficient implementation might work with ListState as well. > > Jan > > [1] > > https://docs.google.com/document/d/1ObLVUFsf1NcG8ZuIZE4aVy2RYKx2Ff