Hi Yang, Thanks for your help, that command worked, so we connected a remote debugger and found the root exception was initially a timeout exception from okhttp. The increases you mentioned worked.
Thanks again for all the help! Best, kevin On 2020/06/19 03:46:36, Yang Wang <d...@gmail.com<mailto:d...@gmail.com>> wrote: > Thanks for sharing the DEBUG level log.> > > I carefully check the logs and find that the kubernetes-client discovered> > the> > api server address and token successfully. However, it could not contact> > with> > api server(10.100.0.1:443). Could you check whether you api server is> > configured> > to allow accessing within cluster.> > > I think you could start any pod and tunnel in to run the following command.> > > KUBE_TOKEN=$(</var/run/secrets/kubernetes.io/serviceaccount/token)> > wget -vO- --ca-certificate> > /var/run/secrets/kubernetes.io/serviceaccount/ca.crt --header> > "Authorization: Bearer $KUBE_TOKEN"> > https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_PORT_443_TCP_PORT/api> > > BTW, what's your kubernetes version? And i am not sure whether increasing> > the timeout> > could help.> > > -Dcontainerized.master.env.KUBERNETES_REQUEST_TIMEOUT=60000> > -Dcontainerized.master.env.KUBERNETES_CONNECTION_TIMEOUT=60000> > > > Best,> > Yang> > > > Yang Wang <da...@gmail.com<mailto:da...@gmail.com>> 于2020年6月16日周二 下午12:00写道：> > > > Hi Kevin,> > >> > > Sorry for not notice your last response.> > > Could you share you full DEBUG level jobmanager logs? I will try to figure> > > out> > > whether it is a issue of Flink or K8s. Because i could not reproduce your> > > situation> > > with my local K8s cluster.> > >> > >> > > Best,> > > Yang> > >> > > Yang Wang <da...@gmail.com<mailto:da...@gmail.com>> 于2020年6月8日周一 上午11:02写道：> > >> > >> Hi Kevin,> > >>> > >> It may because the characters length limitation of K8s(no more than> > >> 63)[1]. So the pod> > >> name could not be too long. I notice that you are using the client> > >> automatic generated> > >> cluster-id. It may cause problem and could you set a meaningful> > >> cluster-id for your Flink> > >> session? For example,> > >>> > >> kubernetes-session.sh ... -Dkubernetes.cluster-id=my-flink-k8s-session> > >>> > >> This behavior has been improved in Flink 1.11 to check the length in> > >> client side before submission.> > >>> > >> If it still could not work, could you share your full command and> > >> jobmanager logs? It will help a lot> > >> to find the root cause.> > >>> > >>> > >> [1].> > >> https://stackoverflow.com/questions/50412837/kubernetes-label-name-63-character-limit> > >>> > >>> > >> Best,> > >> Yang> > >>> > >> kb <ke...@comcast.com<mailto:ke...@comcast.com>> 于2020年6月6日周六 上午1:00写道：> > >>> > >>> Thanks Yang for the suggestion, I have tried it and I'm still getting the> > >>> same exception. Is it possible its due to the null pod name? Operation:> > >>> [create] for kind: [Pod] with name: [null] in namespace: [default]> > >>> failed.> > >>>> > >>> Best,> > >>> kevin> > >>>> > >>>> > >>>> > >>> --> > >>> Sent from:> > >>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> > >>>> > >>> > Best, kevin
