Hi Oleksandr, You're right, I was misreading the POM files because Curator is coming in via Flink-Shaded and not directly. So the POM files only contain references directly to Curator as test dependencies. Then I'll defer to others with experience on this topic to chime in.
Cheers, Martijn On Wed, Jan 3, 2024 at 4:36 PM Alex Nitavsky <alexnitav...@gmail.com> wrote: > > Hello Martjin, > > Thank you for reviewing the document. > > I'm under the impression that Flink utilizes Apache Curator to handle > high-availability leader elections for Job Managers with Zookeeper Services > [0]. > In the most recent master version, it seems that Curator is instantiated at > [1] ZooKeeperUtils, and the purpose of invoking this code for non-testing > purposes is [2] and [3]. As these parameters are passed through the Builder > pattern, I find myself unsure about how to convey them through the > flink-conf.yaml file. > > Your insights and guidance on this matter would be greatly appreciated. > > Kind regards > Oleksandr > > [0] > https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/ha/zookeeper_ha/ > [1] > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/util/ZooKeeperUtils.java#L177 > [2] > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/HighAvailabilityServicesUtils.java#L59 > [3] > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/HighAvailabilityServicesUtils.java#L86 > > On Thu, Dec 28, 2023 at 1:07 PM Martijn Visser <martijnvis...@apache.org> > wrote: >> >> Hi Oleksandr, >> >> The FLIP talks about Curator, but outside of flink test utils, the >> usage of Curator is only for test purposes. I don't think there's >> anything preventing you right now from providing these additional >> parameters as values in the flink-conf.yaml ? >> >> Best regards, >> >> Martijn >> >> On Thu, Dec 14, 2023 at 2:21 PM Alex Nitavsky <alexnitav...@gmail.com> wrote: >> > >> > Hi all, >> > >> > I would like to start a discussion thread for: *FLIP-402: Extend ZooKeeper >> > Curator configurations *[1] >> > >> > * Problem statement * >> > Currently Flink misses several Apache Curator configurations, which could >> > be useful for Flink deployment with ZooKeeper as HA provider. >> > >> > * Proposed solution * >> > We have inspected all possible options for Apache Curator and proposed >> > those which could be valuable for Flink users: >> > >> > - high-availability.zookeeper.client.authorization [2] >> > - high-availability.zookeeper.client.maxCloseWaitMs [3] >> > - high-availability.zookeeper.client.simulatedSessionExpirationPercent [4] >> > >> > The proposed way is to reflect those properties into Flink configuration >> > options for Apache ZooKeeper. >> > >> > Looking forward to your feedback and suggestions. >> > >> > Kind regards >> > Oleksandr >> > >> > [1] >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-402%3A+Extend+ZooKeeper+Curator+configurations >> > [2] >> > https://curator.apache.org/apidocs/org/apache/curator/framework/CuratorFrameworkFactory.Builder.html#authorization(java.lang.String,byte%5B%5D) >> > [3] >> > https://curator.apache.org/apidocs/org/apache/curator/framework/CuratorFrameworkFactory.Builder.html#maxCloseWaitMs(int) >> > [4] >> > https://curator.apache.org/apidocs/org/apache/curator/framework/CuratorFrameworkFactory.Builder.html#simulatedSessionExpirationPercent(int)
