I second Yang that it would be a workaround that you set a static ip for EMR master node.
Even in ZooKeeper world reconfig is a new and immature feature since 3.5.3 while Flink uses ZooKeeper 3.4.x. It would be a breaking change if we "just" upgrade zk version but hopefully the Flink community keep digging out a safe upgrade path. Best, tison. Yang Wang <danrtsey...@gmail.com> 于2020年1月22日周三 上午10:34写道： > Hi Aaron, > > I think it is not the responsibility of Flink. Flink uses zookeeper > curator to connect the zk server. If > multiple zk server are specified, it has an automatic retry mechanism. > However, your problem is ip address will change when the EMR instance > restarts. Currently, Flink > can not support dynamically loading configuration. One quick solution is > to use a static ip for EMR > master node[1]. > > > Best, > Yang > > > [1]. > https://aws.amazon.com/premiumsupport/knowledge-center/static-private-ip-master-node-emr/?nc1=h_ls > > Aaron Langford <aaron.langfor...@gmail.com> 于2020年1月22日周三 上午1:48写道： > >> Hello Flink Community, >> >> I'm working on a HA setup of Flink 1.8.1 on AWS EMR and have some >> questions about how Flink interacts with Zookeeper when one of the servers >> in the quorum specified in flink-conf.yaml goes down and is replaced by a >> machine with a new IP address. >> >> Currently, I configure high-availability.zookeeper.quorum to be the IP >> addresses of the 3 master nodes of the EMR cluster, as this is what AWS >> does to enable a highly available YARN setup. >> >> EMR master nodes may go down entirely and need to be replaced by a >> machine with a different instance IP address. I will almost certainly need >> to perform a rolling configuration update to account for this. But will I >> need to restart flink for this to take effect? Is there a way to >> dynamically reload these configs when they change? >> >> Aaron >> >
