It works now. Job manager is able to start. But now, I have run into another issue.
It seems job manager is trying to create configmap in default namespace and namespace/service account where I run job manager does not have access to configmap GET at: https://X.X.X.X/api/v1/namespaces/default/configmaps/dev-cluster-restserver-leader. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. configmaps "dev-cluster-restserver-leader" is forbidden: User "system:serviceaccount:apps-dev:apps" cannot get resource "configmaps" in API group "" in the namespace "default". Is there a workaround apart from adding permission? Can leader configmap be created in the same namespace as Job Manager? On Wed, Jan 20, 2021 at 7:42 PM Yang Wang <danrtsey...@gmail.com> wrote: > You do not need to put flink-kubernetes_2.12-1.12.1.jar under the plugin > directory. Only the S3 fs jar > needs to be put there. > > I think I found the root cause. It seems your flink-dist is still 1.11. > Do you want to use the image 1.11 and enable the K8s HA at the same time? > I think it could not work. Because the HA related codes have been > refactored from release 1.12. > Even though you copy the flink-kubernetes_2.12-1.12.1.jar to > /opt/flink/lib and make the KubernetesHaServicesFactory class > could be resolved. It will encounter other issues after then. > > Could you please try with the latest Flink 1.12 image? > docker pull apache/flink:1.12.1 > > Best, > Yang > > Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月21日周四 上午8:05写道： > >> I did move to 1.12.1 version and also ensured that docker has kubernetes >> jar file in the right location, i.e. >> /opt/flink/plugins/s3-fs-presto/flink-kubernetes_2.12-1.12.1.jar >> >> But job manager is still not able to find the class >> >> 2021-01-21 00:00:49,376 ERROR >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not >> start cluster entrypoint StandaloneApplicationClusterEntryPoint. >> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to >> initialize the cluster entrypoint StandaloneApplicationClusterEntryPoint. >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:190) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:520) >> [flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:94) >> [flink-dist_2.12-1.11.2.jar:1.11.2] >> Caused by: org.apache.flink.util.FlinkException: Could not instantiate >> class >> 'org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory' >> of type >> 'org.apache.flink.runtime.highavailability.HighAvailabilityServicesFactory'. >> Please make sure that this class is on your class path. >> at >> org.apache.flink.util.InstantiationUtil.instantiate(InstantiationUtil.java:352) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.loadCustomHighAvailabilityServicesFactory(HighAvailabilityServicesUtils.java:263) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createCustomHAServices(HighAvailabilityServicesUtils.java:246) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:126) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:306) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:269) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:211) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:172) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> at >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:171) >> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >> ... 2 more >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >> at jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) ~[?:?] >> at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown >> Source) ~[?:?] >> at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?] >> >> On Mon, Jan 18, 2021 at 7:52 PM Yang Wang <danrtsey...@gmail.com> wrote: >> >>> Class >>> "org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory" >>> is introduced from 1.12.0 version. >>> You could try with the latest version 1.12.1[1]. >>> >>> Will that jar file need to be copied under >>>> /opt/flink/plugins/s3-fs-presto folder? >>> >>> Yes, you need to copy the s3 fs implementation to plugin directory. >>> An alternative is you could set the environment to enable the plugin[1]. >>> >>> [1]. >>> https://hub.docker.com/r/apache/flink/tags?page=1&ordering=last_updated&name=1.12.1 >>> [2]. >>> https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#using-filesystem-plugins >>> >>> Best, >>> Yang >>> >>> Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月18日周一 下午11:15写道： >>> >>>> Hi, >>>> I am not able to identify maven artifact that will have >>>> implementation for this class >>>> >>>> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >>>> >>>> >>>> I am using info in this link to test out HA implementation >>>> >>>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/ha/kubernetes_ha.html >>>> >>>> Please point me to the right artifact. Also, I plan to use S3 bucket as >>>> storageDir. So, will that jar file need to be copied under >>>> /opt/flink/plugins/s3-fs-presto folder? >>>> >>>> Thanks >>>> Ashish >>>> >>>
