Manish Jain created FLINK-38583:
-----------------------------------
Summary: Flink job does not restart when environment variable
changes.
Key: FLINK-38583
URL: https://issues.apache.org/jira/browse/FLINK-38583
Project: Flink
Issue Type: Bug
Components: Deployment / Kubernetes, Kubernetes Operator
Affects Versions: 1.12.1
Environment: Dev/Prod
Reporter: Manish Jain
We are using flink k8s operator to run and manage Flink cluster on our Azure
k8s environment.
Our jobs are deployed in ApplicationMode, and to connect to external services,
we use some secrets which are passed as environment variable for the JobManager
and TaskManager Pods.
{code:java}
podTemplate:
spec:
containers:
- name: flink-main-container # this name is fixed and should not be
changed
ports:
- name: metrics
containerPort: 9999
env:
- name: OAUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: "{{ .oauthClientSecretRef }}"
key: "password" {code}
As part of our deployment, we have a script in place, which rotates the secret
every x days.
In our k8s cluster, we are having a azure secret mounter component, which
observes the changes in secret, and then updates them in the k8s environment.
We are also using a reloader app, which restarts the pods when a config
changes. On secret change, all other pods get restarted automatically (Spring
boot pods, Prometheus, grafana, kafka), but the flink Pods, do not restart.
This is leading to a problem, because now, we need to manuall restart the pod
when the secret changes. Not doing so, leads to Authn issues.
Is this a known issue and is it possible to ensure that once the secret
referenced by the JobManager and TaskManager is updated, then the pod should
restart?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
