Re: Data Stream countWindow followed by keyBy does not preserve time order

2021-11-04 Thread Yan Shen
Thanks for the advice. I created this issue: https://issues.apache.org/jira/browse/FLINK-24767 On Thu, Nov 4, 2021 at 2:44 PM Guowei Ma wrote: > Hi Yan > After a second thought I think you are right, the downstream operator > should keep the order of the same key from the same upstream. So feel

Re: Data Stream countWindow followed by keyBy does not preserve time order

2021-11-03 Thread Guowei Ma
Hi Yan After a second thought I think you are right, the downstream operator should keep the order of the same key from the same upstream. So feel free to open a jira. Best, Guowei On Wed, Nov 3, 2021 at 7:30 PM Yan Shen wrote: > Hi, > > It will complicate things a lot if we cannot assume input

Re: Data Stream countWindow followed by keyBy does not preserve time order

2021-11-03 Thread Yan Shen
Hi, It will complicate things a lot if we cannot assume input order of any operator after a keyBy. So far I only have the problem with countWindow which I seem to be able to avoid by writing my own stateful KeyedProcess. Are there other operators which might cause the same problem? The other alte

Re: Data Stream countWindow followed by keyBy does not preserve time order

2021-11-03 Thread Guowei Ma
Hi, Yan I do not think it is a bug. Maybe we could not assume the input's order of an operator simply. Best, Guowei On Wed, Nov 3, 2021 at 3:10 PM Yan Shen wrote: > Yes, it does not happen in streaming mode. Is this considered a bug or is > it by design? > > Thanks! > > On Wed, Nov 3, 2021 at 1

Re: Data Stream countWindow followed by keyBy does not preserve time order

2021-11-03 Thread Yan Shen
Yes, it does not happen in streaming mode. Is this considered a bug or is it by design? Thanks! On Wed, Nov 3, 2021 at 1:58 PM Guowei Ma wrote: > Hi > > I did not run your program directly, but I see that you are now using the > Batch execution mode. I suspect it is related to this, because in

Re: Data Stream countWindow followed by keyBy does not preserve time order

2021-11-02 Thread Guowei Ma
Hi I did not run your program directly, but I see that you are now using the Batch execution mode. I suspect it is related to this, because in the Batch execution mode FLINK will "sort" the Key (this might be an unstable sort). So would you like to experiment with the results of running with Strea