Re:State life-cycle for different state-backend implementations

2018-06-09 Thread sihua zhou
Hi Rinat, I think there is one configuration {{state.checkpoints.num-retained}} to control the maximum number of completed checkpoints to retain, the default value is 1. So the risk you mentioned should not happen. Refer to https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config

Re: [DISCUSS] Flink 1.6 features

2018-06-09 Thread sihua zhou
Hi Stephan, Thanks very much for your response! That gave me the confidence to continue to work on the Elastic Filter. But even though we have implemented it(based on 1.3.2) and used it on production for a several months, If there's one commiter is willing to guide me(since it's not a very tri

Re:Checkpointing Large State to S3

2018-06-09 Thread sihua zhou
Hi Gregory, could you share the TaskManager's log with us? It would be helpful to diagnost the problem. And which version are you using? Best, Sihua On 06/7/2018 06:42,Gregory Fee wrote: Hello Everyone! I am running some streaming Flink jobs using SQL and the table api. I enabled incremen

Unsubscribe

2018-06-09 Thread Flashacid