Re: checkpointing when yarn session crashed

2019-04-08 Thread Guowei Ma
Could you give more details? Such as which flink version do you use? which Statebackend do you use? Does there has any successful checkpoint? and so on.. I can't reproduce your problem. (I used BucketingSinkTestProgram(enable external checkpoint) + Flink 1.7.2 and default StateBackend ) Best, Gu

checkpointing when yarn session crashed

2019-04-07 Thread Shengnan YU
Hi everyone: Recently I am doing a bucketingsink (to hdfs) job by flink on yarn. However I found that if the yarn session crashed, or I manually killed the yarn session, the file on hdfs does not rename to .pending state and the latest checkpoint does not have _metadata. Therefore I cannot resu