Re: Flink CLI properties with HA

2018-07-18 Thread Sampath Bhat
Vino, I'm not getting any error but my suspicion was that if I dont specify this `high-availability.storageDir` property in flink CLI side then the CLI will not be able to submit job to flink cluster(HA enabled). But if provide this property in CLI side the job submission will be successful even th

Re: Flink CLI properties with HA

2018-07-17 Thread Till Rohrmann
Hi Sampath, technically the client does not need to know the `high-availability.storageDir` to submit a job. However, due to how we construct the ZooKeeperHaServices it is still needed. The reason behind this is that we use the same services for the server and the client. Thus, the implementation

Re: Flink CLI properties with HA

2018-07-17 Thread vino yang
Hi Sampath, It seems Flink CLI for standalone would not access *high-availability.storageDir.* What's the exception stack trace in your environment? Thanks, vino. 2018-07-17 15:08 GMT+08:00 Sampath Bhat : > Hi vino > > Should the flink CLI have access to the path mentioned in > *high-availabil

Re: Flink CLI properties with HA

2018-07-17 Thread Sampath Bhat
Hi vino Should the flink CLI have access to the path mentioned in *high-availability.storageDir*? If my flink cluster is on set of machines and i submit my job from flink CLI from another independent machine by giving necessary details will the CLI try to access *high-availability.storageDir *path

Re: Flink CLI properties with HA

2018-07-16 Thread vino yang
Hi Sampath, Flink CLI need to retrieve the JobManager leader address, so it need to access the HA specific configuration. Because if based on Zookeeper to implement the HA, the leader address information will fetch from Zookeeper. The main use of config item *high-availability.storageDir* is sto