Re: Random checkpoint failures with timeouts

2021-11-23 Thread Yun Gao
Hi Dineth, In the UI of flink there is pages for details for the checkpoints[1], could you have a look this UI to see which part of checkpoint took long time~? Best, Yun [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/monitoring/checkpoint_monitoring/

Random checkpoint failures with timeouts

2021-11-23 Thread Dineth Kariyawasam
Checkpoint fails randomly with a timeout. Many times this happens when there are no other events coming into flink (at night). Most of our incoming data is during the daytime, and at night there are usually no events. Many of these failures have been at night. We had set a checkpoint timeout of 2 m