Hey Hironori, the storage directories (recovery.zookeeper.storageDir, state.backend.fs.checkpointdir) can stay the same I think (either random or jobID-specific sub folders should be created there). The ZooKeeper root path (recovery.zookeeper.path.root) needs to be unique per cluster for HA.
If you upgrade to the to be released 1.1 (vote just passed, binaries are being uploaded) this will be set automatically for YARN. You can also specify it via the new CLI parameter -z <UNIQUE-NAME> (this sets recovery.zookeeper.path.root). Hope this helps. Ufuk On Thu, Aug 4, 2016 at 2:53 AM, Hironori Ogibayashi <ogibaya...@gmail.com> wrote: > Hello, > > I have a question about Zookeeper or HDFS paths in case of running > Flink on YARN. > > In my understanding, when I run multiple Flink cluster using the same > zookeeper/HDFS, I have to specify different paths, as e.g. > state.backend.fs.checkpointdir, > recovery.zookeeper.path.root etc. > > If I run multiple Flink cluster job on YARN, and want to use checkpoint or > JobManager HA, do I need to specify different paths for each cluster/job? or > does YARN handle this nicely? > > Regards, > Hironori Ogibayashi
