Hi Xiao, Is istio or something similar deployed to the K8S cluster?
John On Tue, May 17, 2022 at 4:26 PM Xiao Ma <xia...@geotab.com> wrote: > loop in > *Xiao Ma* > *Geotab* > Software Developer, Data Engineering | B.Sc, M.Sc > Direct +1 (416) 836 - 3541 > Toll-free +1 (877) 436 - 8221 > Visit www.geotab.com > Twitter <https://twitter.com/geotab> | Facebook > <https://www.facebook.com/Geotab> | YouTube > <https://www.youtube.com/user/MyGeotab> | LinkedIn > <https://www.linkedin.com/company/geotab/> > > > ---------- Forwarded message --------- > From: Xiao Ma <xia...@geotab.com> > Date: Tue, May 17, 2022 at 4:18 PM > Subject: Re: Question of Flink Operator Application Cluster Deployment > To: Őrhidi Mátyás <matyas.orh...@gmail.com> > > > Fyi, I didn't manually mount the service account token into the job pod. > It is automatically mounted into the pod, with the "bound service account > token volume". I also found that the fabric8 cannot read the service > account token if it is the "bound service account token volume". link: > https://github.com/fabric8io/kubernetes-client/issues/2271 > > Thank you very much. > > Best, > *Xiao Ma* > *Geotab* > Software Developer, Data Engineering | B.Sc, M.Sc > Direct +1 (416) 836 - 3541 > Toll-free +1 (877) 436 - 8221 > Visit www.geotab.com > Twitter <https://twitter.com/geotab> | Facebook > <https://www.facebook.com/Geotab> | YouTube > <https://www.youtube.com/user/MyGeotab> | LinkedIn > <https://www.linkedin.com/company/geotab/> > > > On Tue, May 17, 2022 at 10:55 AM Xiao Ma <xia...@geotab.com> wrote: > >> Hi Őrhidi, >> >> Thank you very much for the help. >> >> The attached are flink-operator yaml files and the application job yaml >> file. >> >> Best, >> *Xiao Ma* >> *Geotab* >> Software Developer, Data Engineering | B.Sc, M.Sc >> Direct +1 (416) 836 - 3541 >> Toll-free +1 (877) 436 - 8221 >> Visit www.geotab.com >> Twitter <https://twitter.com/geotab> | Facebook >> <https://www.facebook.com/Geotab> | YouTube >> <https://www.youtube.com/user/MyGeotab> | LinkedIn >> <https://www.linkedin.com/company/geotab/> >> >> >> On Tue, May 17, 2022 at 12:22 AM Őrhidi Mátyás <matyas.orh...@gmail.com> >> wrote: >> >>> You don't have to mount the service account explicitly, this should >>> be auto-mounted for you. Please share your (redacted) yamls for the RBAC >>> configs ( >>> https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/rbac/#cluster-scoped-flink-operator-with-jobs-running-in-other-namespaces) >>> and your deployment yaml, we could probably spot what's missing. >>> >>> Best, >>> Matyas >>> >>> On Tue, May 17, 2022 at 5:37 AM Xiao Ma <xia...@geotab.com> wrote: >>> >>>> Hi Flink Community, >>>> >>>> First of all, I would like to express my great thankfulness about the >>>> flink operator on Kubernetes. It is a new door to help us deploy the Flink >>>> application on top of the K8s. >>>> >>>> Our team is currently doing the Application cluster deployment through >>>> the operator. We have set up the service account as "flink-operator" and >>>> "flink", with the roles and rolebindings. However, after the job yaml is >>>> submitted to the api-server and the pod is created, the resources manager >>>> cannot be created because this error log: >>>> ==== >>>> 2022-05-17 02:37:22,293 WARN io.fabric8.kubernetes.client.Config >>>> [] - Error reading service account token from: >>>> [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring. >>>> 2022-05-17 02:37:22,308 WARN io.fabric8.kubernetes.client.Config >>>> [] - Error reading service account token from: >>>> [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring. >>>> 2022-05-17 02:37:25,699 INFO >>>> org.apache.flink.runtime.jobmaster.JobMaster [] - >>>> Connecting to ResourceManager akka.tcp://fl...@flink-application-job.bip >>>> :6123/user/rpc/resourcemanager_*(00000000000000000000000000000000) >>>> 2022-05-17 02:37:26,094 WARN >>>> io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener [] - >>>> Exec Failure: HTTP 403, Status: 403 - pods is forbidden: User >>>> "system:anonymous" cannot watch resource "pods" in API group "" in the >>>> namespace "xxxxxxxxx" >>>> ==== >>>> >>>> It looks like the jobmanager pod cannot fetch the "flink" service >>>> account token and cannot communicate with api-server, though I have created >>>> the "flink" service account and set up "serviceAccount" config in the job >>>> template. >>>> ==== >>>> >>>> apiVersion: flink.apache.org/v1beta1 >>>> kind: FlinkDeployment >>>> metadata: >>>> name: flink-application-job >>>> spec: >>>> image: flink:1.15.0-scala_2.12-java11 >>>> flinkVersion: v1_15 >>>> flinkConfiguration: >>>> taskmanager.numberOfTaskSlots: "2" >>>> jobmanager.rpc.address: flink-jobmanager >>>> serviceAccount: flink >>>> >>>> ==== >>>> >>>> The below shows the volumeMounts in the pod. The service account is >>>> mounted through the "bound service account token volume". Is it desirable? >>>> ==== >>>> Mounts: >>>> /opt/flink/conf from flink-config-volume (rw) >>>> /opt/flink/log from flink-logs (rw) >>>> /opt/flink/pod-template from pod-template-volume (rw) >>>> /var/run/secrets/kubernetes.io/serviceaccount from >>>> kube-api-access-f69zl (ro) >>>> ==== >>>> >>>> This issue has blocked our progress for several days so if there are >>>> any possible thoughts, we really appreciate it! >>>> >>>> Thank you very much and I'm looking forward to your reply. >>>> >>>> >>>> Best, >>>> *Xiao Ma* >>>> *Geotab* >>>> Software Developer, Data Engineering | B.Sc, M.Sc >>>> Direct +1 (416) 836 - 3541 >>>> Toll-free +1 (877) 436 - 8221 >>>> Visit www.geotab.com >>>> Twitter <https://twitter.com/geotab> | Facebook >>>> <https://www.facebook.com/Geotab> | YouTube >>>> <https://www.youtube.com/user/MyGeotab> | LinkedIn >>>> <https://www.linkedin.com/company/geotab/> >>>> >>>
