You do not need to put flink-kubernetes_2.12-1.12.1.jar under the plugin directory. Only the S3 fs jar needs to be put there.
I think I found the root cause. It seems your flink-dist is still 1.11. Do you want to use the image 1.11 and enable the K8s HA at the same time? I think it could not work. Because the HA related codes have been refactored from release 1.12. Even though you copy the flink-kubernetes_2.12-1.12.1.jar to /opt/flink/lib and make the KubernetesHaServicesFactory class could be resolved. It will encounter other issues after then. Could you please try with the latest Flink 1.12 image? docker pull apache/flink:1.12.1 Best, Yang Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月21日周四 上午8:05写道： > I did move to 1.12.1 version and also ensured that docker has kubernetes > jar file in the right location, i.e. > /opt/flink/plugins/s3-fs-presto/flink-kubernetes_2.12-1.12.1.jar > > But job manager is still not able to find the class > > 2021-01-21 00:00:49,376 ERROR > org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not > start cluster entrypoint StandaloneApplicationClusterEntryPoint. > org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to > initialize the cluster entrypoint StandaloneApplicationClusterEntryPoint. > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:190) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:520) > [flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:94) > [flink-dist_2.12-1.11.2.jar:1.11.2] > Caused by: org.apache.flink.util.FlinkException: Could not instantiate > class > 'org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory' > of type > 'org.apache.flink.runtime.highavailability.HighAvailabilityServicesFactory'. > Please make sure that this class is on your class path. > at > org.apache.flink.util.InstantiationUtil.instantiate(InstantiationUtil.java:352) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.loadCustomHighAvailabilityServicesFactory(HighAvailabilityServicesUtils.java:263) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createCustomHAServices(HighAvailabilityServicesUtils.java:246) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:126) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:306) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:269) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:211) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:172) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > at > org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:171) > ~[flink-dist_2.12-1.11.2.jar:1.11.2] > ... 2 more > Caused by: java.lang.ClassNotFoundException: > org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory > at jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) ~[?:?] > at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown > Source) ~[?:?] > at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?] > > On Mon, Jan 18, 2021 at 7:52 PM Yang Wang <danrtsey...@gmail.com> wrote: > >> Class >> "org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory" >> is introduced from 1.12.0 version. >> You could try with the latest version 1.12.1[1]. >> >> Will that jar file need to be copied under >>> /opt/flink/plugins/s3-fs-presto folder? >> >> Yes, you need to copy the s3 fs implementation to plugin directory. >> An alternative is you could set the environment to enable the plugin[1]. >> >> [1]. >> https://hub.docker.com/r/apache/flink/tags?page=1&ordering=last_updated&name=1.12.1 >> [2]. >> https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#using-filesystem-plugins >> >> Best, >> Yang >> >> Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月18日周一 下午11:15写道： >> >>> Hi, >>> I am not able to identify maven artifact that will have >>> implementation for this class >>> >>> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >>> >>> >>> I am using info in this link to test out HA implementation >>> >>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/ha/kubernetes_ha.html >>> >>> Please point me to the right artifact. Also, I plan to use S3 bucket as >>> storageDir. So, will that jar file need to be copied under >>> /opt/flink/plugins/s3-fs-presto folder? >>> >>> Thanks >>> Ashish >>> >>
