I’ve been working on migrating our Flink cluster to zookeeper SSL (this on Flink 1.12) but ran into a missing dependency issue.
Steps to reproduce: * Replace flink-shaded-zookeeper-3.4.14.jar in the lib folder with flink-shaded-zookeeper-3.5.6.jar from the opt folder * Add zookeeper SSL config to env.java.opts: -Dzookeeper.client.secure=true -Dzookeeper.clientCnxnSocket=org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxnSocketNetty -Dzookeeper.ssl.trustStore.location=… * Change high-availability.zookeeper.quorum to SSL port When I start Flink, I get the following exception: Shutting StandaloneSessionClusterEntrypoint down with application status FAILED. Diagnostics java.lang.NoClassDefFoundError: io/netty/channel/ChannelHandler\n\tat java.lang.Class.forName0(Native Method)\n\tat java.lang.Class.forName(Class.java:264)\n\tat org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:3063)\n\tat org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:883)\n\tat org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:801)\n\tat org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:950)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:213)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:101)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:57)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.ConnectionState.reset(ConnectionState.java:204)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.ConnectionState.start(ConnectionState.java:111)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.CuratorZookeeperClient.start(CuratorZookeeperClient.java:237)\n\tat org.apache.flink.shaded.curator4.org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:338)\n\tat org.apache.flink.runtime.util.ZooKeeperUtils.startCuratorFramework(ZooKeeperUtils.java:162)\n\tat org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:118)\n\tat org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:338)\n\tat org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:296)\n\tat org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:224)\n\tat org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:178)\n\tat org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)\n\tat org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:175)\n\tat org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:585)\n\tat org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:59)\nCaused by: java.lang.ClassNotFoundException: io.netty.channel.ChannelHandler\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:382)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:418)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:351)\n\t... The pom explicitly excludes netty dependencies as they are “only required for ZK servers, not clients” (see https://github.com/apache/flink-shaded/blob/master/flink-shaded-zookeeper-parent/flink-shaded-zookeeper-35/pom.xml#L47). Looks like this is not correct if you want to use SSL. Adding jars for netty-handler and netty-transport-native-epoll to the lib folder addressed this issue. Perhaps this could be addressed in the next release for flink-shaded? Thanks, Peter Westermann Team Lead – Realtime Analytics [cidimage001.jpg@01D78D4C.C00AC080] peter.westerm...@genesys.com<mailto:peter.westerm...@genesys.com> [cidimage001.jpg@01D78D4C.C00AC080] [cidimage002.jpg@01D78D4C.C00AC080]<http://www.genesys.com/>
