Noa Ramon created FLINK-41403:
---------------------------------
Summary: Blue/Green child FlinkDeployments do not inherit the
parent's labels, so a label-selector-scoped operator never reconciles them
Key: FLINK-41403
URL: https://issues.apache.org/jira/browse/FLINK-41403
Project: Flink
Issue Type: Bug
Affects Versions: kubernetes-operator-1.15.0
Reporter: Noa Ramon
When the Blue/Green controller builds a child \{{FlinkDeployment}}, it replaces
the child's labels with the Blue/Green type marker alone
(\{{BlueGreenUtils.java:382}} on \{{main}} at 43719c98):
{code:java}
flinkDeploymentMeta.setLabels(
Map.of(BlueGreenDeploymentType.LABEL_KEY,
blueGreenDeploymentType.toString()));
{code}
None of the parent \{{FlinkBlueGreenDeployment}}'s labels reach the children.
With \{{kubernetes.operator.label.selector}} set, the operator reconciles the
parent (which matches the selector) and creates the child. The child does not
match the selector, so \{{FlinkDeploymentController}} never reconciles it: no
status, no events, no logs. The Blue/Green controller polls the child as "not
ready yet" until \{{kubernetes.operator.bluegreen.abort.grace-period}} expires,
and the parent goes \{{FAILING}}. A first deployment never starts and every
transition aborts. The only trace of the child being seen at all is the JOSDK
reconciliation-count metric.
Selector-scoped operators are a common setup (several operator installations in
one cluster, a shared namespace, label-based ownership). In such a setup
Blue/Green cannot work unless the children are labelled by hand after creation.
h4. Steps to reproduce
# Start the operator with \{{kubernetes.operator.label.selector: "team=a"}}.
# Apply a \{{FlinkBlueGreenDeployment}} with \{{metadata.labels: \{team: a\}}}.
# Observed: the parent is reconciled and creates \{{<name>-blue}} carrying only
the \{{blue-green-deployment-type}} label. \{{<name>-blue}} never gets a
status. After the abort grace period the parent is \{{FAILING}}.
# Expected: \{{<name>-blue}} carries \{{team=a}}, is reconciled, becomes ready,
and the parent reaches \{{ACTIVE_BLUE}}.
h4. Proposed fix
Copy the parent's labels onto the child, then set the Blue/Green type marker on
top:
{code:java}
Map<String, String> labels = new HashMap<>(parentLabels);
labels.remove(CanaryResourceManager.CANARY_LABEL);
labels.put(BlueGreenDeploymentType.LABEL_KEY,
blueGreenDeploymentType.toString());
{code}
{\{flink.apache.org/canary}} is deliberately not inherited: a child carrying it
would be skipped by \{{FlinkDeploymentController}} at its first branch, which
reproduces the same silent hang whenever the parent is itself a canary
resource. Inheriting the remaining labels also keeps identity and
cost-attribution labels consistent between a parent and its children.
This change has been running on a selector-scoped operator: children are
created already carrying the selector labels, are reconciled immediately, and
transitions complete without manual labelling.
A patch against \{{main}} is ready, with a unit test
(\{{verifyChildInheritsParentLabelsWithoutTheCanaryMarker}}) that fails without
the fix. I will open the PR once this ticket exists.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
