Re: Replacing a server in Zookeeper Quorum

2020-01-21 Thread tison
Good to know :-) Best, tison. Aaron Langford 于2020年1月22日周三 上午10:44写道: > My apologies, I ended up resolving this through experimentation. AWS > replaces master nodes with the same internal DNS names, so configurations > need not be changed. > > Aaron > > > On Tue, Jan 21, 2020, 6:33 PM Yang Wan

Re: Replacing a server in Zookeeper Quorum

2020-01-21 Thread Aaron Langford
My apologies, I ended up resolving this through experimentation. AWS replaces master nodes with the same internal DNS names, so configurations need not be changed. Aaron On Tue, Jan 21, 2020, 6:33 PM Yang Wang wrote: > Hi Aaron, > > I think it is not the responsibility of Flink. Flink uses zoo

Re: Replacing a server in Zookeeper Quorum

2020-01-21 Thread tison
I second Yang that it would be a workaround that you set a static ip for EMR master node. Even in ZooKeeper world reconfig is a new and immature feature since 3.5.3 while Flink uses ZooKeeper 3.4.x. It would be a breaking change if we "just" upgrade zk version but hopefully the Flink community kee

Re: Replacing a server in Zookeeper Quorum

2020-01-21 Thread Yang Wang
Hi Aaron, I think it is not the responsibility of Flink. Flink uses zookeeper curator to connect the zk server. If multiple zk server are specified, it has an automatic retry mechanism. However, your problem is ip address will change when the EMR instance restarts. Currently, Flink can not support

Replacing a server in Zookeeper Quorum

2020-01-21 Thread Aaron Langford
Hello Flink Community, I'm working on a HA setup of Flink 1.8.1 on AWS EMR and have some questions about how Flink interacts with Zookeeper when one of the servers in the quorum specified in flink-conf.yaml goes down and is replaced by a machine with a new IP address. Currently, I configure high-