Hi, thank you for the answer. I think I understand.
In my uses case I have to keep the order of events for each key, but I dont have to process keys in the same order that I received them. On one point of my pipeline I'm also using a SessionWindow. My Flink environment has operator chaining enabled. I woudl say, that some of my operators can be chained. My pipeline is (each point is an operator after Flink's operator chainign mechanism) 1. ActiveMQ connector + mapper, all with parallelism 1 (btw I'm using a org.apache.bahir connector for Active MQ which does not support parallelism bigger than 1) 2. Enrichment, where Im using AsyncDataStream.unorderedWait with parallelism 5. 3. Event split based on some criteria (not key by) that dispatches my stream into two "sub streams" 4. Both substreams are keyed 4a. SubStream "A" has a session window applied - parallelism 6. 4b. Substream "B" has no windowing, no aggregation, but has a business logic for witch order of events matters. - parallelism 6 5. Sink for both streams. If I understand you and documentation correctly, Redistributing will forward messages keeping the order for a key, but events between keys can be delivered in a different order. "So in this example, the ordering within each key is preserved, but the parallelism does introduce non-determinism regarding the order in which the aggregated results for different keys arrive at the sink." Then I could use a keyBy at the pipeline beginning, just after point 1. But to use Window in point 4a and my process function in 4b I need to have a keyedStream. I'm using a KeyedProcessFunction there. What my options with this? P.S. Regarding the operator chaining, I'm aware that there is an API that allows me to model which operators should be chained theatergoer and which not even if they have the same parallelism level. Thanks, Krzysztof -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
