If you have created the role binding "flink-role-binding-default" successfully, then it should not be the RBAC issue.
It seems that kubernetes-client in JobManager pod could not contact to K8s apiserver due to okhttp issue with java 8u252. Could you add the following config option to disable http2? You could find more information here[1]. kubernetes-session.sh ... -Dcontainerized.master.env.HTTP2_DISABLE=true [1]. https://github.com/fabric8io/kubernetes-client/issues/2212 Best, Yang kb <kevin_bohin...@comcast.com> 于2020年6月4日周四 下午11:40写道： > Thanks! > > I do not see any pods of the form `flink-taskmanager-1-1`, so I tried the > exec suggestion. > The logs are attached below. Is there a quick RBAC check I could perform? I > followed the command on the docs page linked (kubectl create > clusterrolebinding flink-role-binding-default --clusterrole=edit > --serviceaccount=default:default). > > 2020-06-04 15:34:04,711 INFO > org.apache.flink.kubernetes.KubernetesResourceManager - Requesting > new TaskManager pod with <1728,1.0>. Number pending requests 1. > 2020-06-04 15:34:04,712 INFO > org.apache.flink.kubernetes.KubernetesResourceManager - TaskManager > flink-cluster-e07a6f7a-8bd1-4306-89f1-a1ff7ea17bf6-taskmanager-1-5994 will > be started with TaskExecutorProcessSpec {cpuCores=1.0, > frameworkHeapSize=128.000mb (134217728 bytes), > frameworkOffHeapSize=128.000mb (134217728 bytes), taskHeapSize=384.000mb > (402653174 bytes), taskOffHeapSize=0 bytes, networkMemSize=128.000mb > (134217730 bytes), managedMemorySize=512.000mb (536870920 bytes), > jvmMetaspaceSize=256.000mb (268435456 bytes), jvmOverheadSize=192.000mb > (201326592 bytes)}. > 2020-06-04 15:34:14,713 ERROR > org.apache.flink.kubernetes.KubernetesResourceManager - Could not > start TaskManager in pod > flink-cluster-e07a6f7a-8bd1-4306-89f1-a1ff7ea17bf6-taskmanager-1-5994. > java.util.concurrent.CompletionException: > io.fabric8.kubernetes.client.KubernetesClientException: Operation: > [create] > for kind: [Pod] with name: [null] in namespace: [default] failed. > at > > java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) > at > > java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) > at > > java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1643) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: io.fabric8.kubernetes.client.KubernetesClientException: > Operation: [create] for kind: [Pod] with name: [null] in namespace: > [default] failed. > at > > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) > at > > io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:331) > at > > io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:324) > at > > org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClient.lambda$createTaskManagerPod$0(Fabric8FlinkKubeClient.java:184) > at > > java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640) > ... 3 more > Caused by: java.net.SocketTimeoutException: timeout > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:656) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:664) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.java:153) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:131) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) > at > > io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) > at > > io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) > at > > io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:110) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) > at > > org.apache.flink.kubernetes.shadded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) > at > > org.apache.flink.kubernetes.shadded.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:254) > at > > org.apache.flink.kubernetes.shadded.okhttp3.RealCall.execute(RealCall.java:92) > at > > io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:411) > at > > io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:372) > at > > io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:241) > at > > io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:798) > at > > io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:328) > ... 6 more > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
