Re: Any issues with reinterpretAsKeyedStream when scaling partitions?

2021-10-24 Thread JING ZHANG
> > My top guess was that the mapping of keys to task managers changes > depending on the operator. E.g. if the same key (regardless of the > operator) goes to the same task manager, then I'd assume it's fine to > support multiple inputs into an operator with reinterpretAsKeyedStream. I agree wit

Not cleanup Kubernetes Configmaps after execution success

2021-10-24 Thread Hua Wei Chen
Hi all, We have Flink jobs run on batch mode and get the job status via JobHandler. onJobExecuted ()[

Re: Any issues with reinterpretAsKeyedStream when scaling partitions?

2021-10-24 Thread Dan Hill
On Sat, Oct 23, 2021 at 9:16 PM JING ZHANG wrote: > I found the following link about this. Still looks applicable. In my >> case, I don't need to do a broadcast join. >> >> https://www.alibabacloud.com/blog/flink-how-to-optimize-sql-performance-using-multiple-input-operators_597839 >> > This is

Re: how to delete all rows one by one in batch execution mode;shutdown cluster after all tasks finished

2021-10-24 Thread Caizhi Weng
Hi! Thanks for the clarification. Flink currently does not have the functionality to "revert all operations till some point". What I would suggest is still to discard the resulting tables and run the pipeline from the point when the filtering logic is changed. If the pipeline has processed some da