Re: Query regarding HA mode and checkpointing

2020-12-03 Thread Yang Wang
Hi Kaushal, Only the state handle pointer is stored in the ZooKeeper node. Since ZooKeeper is built for small data(KB level) storage. The real data will be persisted in the *"* *high-availability.storageDir"*. Note that it should be distributed storage(HDFS, S3, etc.). The ZooKeeper HA service ha

Query regarding HA mode and checkpointing

2020-12-03 Thread Kaushal Raj
Hello, I am new to flink. Have few queries regarding the HA mode with zookeeper and checkpointing. When flink is configured in HA mode with zookeeper, where do the job checkpoints are stored? zookeeper only used for recovering the jobmanager or even checkpoints? what is the significance of *"**hig