Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-11 Thread Yang Wang
After FLINK-10052[1], which was merged in 1.14.0, rolling upgrading ZooKeeper will not affect the running Flink application. 1. https://issues.apache.org/jira/browse/FLINK-10052 Best, Yang Chesnay Schepler 于2021年12月7日周二 下午4:37写道: > Since this is only relevant for 1.15, if you intend to migrat

Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-07 Thread Chesnay Schepler
Since this is only relevant for 1.15, if you intend to migrate to 1.15 close to the release, then somewhere around February. The only resource I could find for migrating Zookeeper is this FAQ: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ On 07/12/2021 04:02, Dongwon Kim w

Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-07 Thread Dongwon Kim
When should I prepare for upgrading ZK to 3.5 or newer? We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs. Just hope that the rolling update is not that painful - any advice on this? Best, Dongwon On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler wrote: > Current users of

Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-06 Thread Yang Wang
FYI: We(Alibaba) are widely using ZooKeeper 3.5.5 for all the YARN and some K8s Flink high-available applications. Best, Yang Chesnay Schepler 于2021年12月7日周二 上午2:22写道: > Current users of ZK 3.4 and below would need to upgrade their Zookeeper > installation that is used by Flink to 3.5+. > > Wh

Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-06 Thread Chesnay Schepler
Current users of ZK 3.4 and below would need to upgrade their Zookeeper installation that is used by Flink to 3.5+. Whether K8s users are affected depends on whether they use ZK or not. If they do, see above, otherwise they are not affected at all. On 06/12/2021 18:49, Arvid Heise wrote: Cou

Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-06 Thread Arvid Heise
Could someone please help me understand the implications of the upgrade? As far as I understood this upgrade would only affect users that have a zookeeper shared across multiple services, some of which require ZK 3.4-? A workaround for those users would be to run two ZKs with different versions, e

Re: [DISCUSS] Drop Zookeeper 3.4

2021-12-06 Thread Chesnay Schepler
ping @users; any input on how this would affect you is highly appreciated. On 25/11/2021 22:39, Chesnay Schepler wrote: I included the user ML in the thread. @users Are you still using Zookeeper 3.4? If so, were you planning to upgrade Zookeeper in the near future? I'm not sure about ZK comp

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-26 Thread Till Rohrmann
According to this SO question [1], it seems that Zk 3.5 clients cannot talk to 3.4 servers. I also tried it out with a local deployment and Flink was not able to start. Newer Zk versions can talk to older Zk servers if no new APIs are used [2]. [1] https://stackoverflow.com/a/61630617/4815083 [2]

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-25 Thread Chesnay Schepler
I included the user ML in the thread. @users Are you still using Zookeeper 3.4? If so, were you planning to upgrade Zookeeper in the near future? I'm not sure about ZK compatibility, but we'd also upgrade Curator to 5.x, which doesn't support ookeeperK 3.4 anymore. On 25/11/2021 21:56, Till

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-25 Thread Till Rohrmann
Should we ask on the user mailing list whether anybody is still using ZooKeeper 3.4 and thus needs support for this version or can a ZooKeeper 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that not a lot of users depend on it but just to make sure that we aren't annoying a lot of o

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-24 Thread Matthias Pohl
Thanks for starting this discussion, Chesnay. +1 from my side. It's time to move forward with the ZK support considering the EOL of 3.4 you already mentioned. The benefits we gain from upgrading Curator to 5.x as a consequence is another plus point. Just for reference on the inconsistent state issu

[DISCUSS] Drop Zookeeper 3.4

2021-11-24 Thread Chesnay Schepler
Hello, I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the default to 3.5 with an opt-in for 3.6. Supporting Zookeeper 3.4 (which is already EOL) prevents us from upgrading Curator to 5.x, which would allow us to properly fix an issue with inconsistent state. It is also require