Dennis-Mircea Ciupitu created FLINK-40380:
---------------------------------------------
Summary: Document the runtime configuration and stop jobs from
overriding operator config through it
Key: FLINK-40380
URL: https://issues.apache.org/jira/browse/FLINK-40380
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Dennis-Mircea Ciupitu
h1. Summary
FLINK-35746 added runtime configuration observation, where the operator reads a
job's effective settings from the Flink REST API and layers them over the
observed configuration. Two follow-ups are needed: the operator documentation
still describes the previous configuration model, and the global job parameters
ingested from the REST response are not filtered.
h1. Documentation is out of date
The {{Internals -> Startup -> Configuration}} section still describes the model
as it was before FLINK-35746, and three statements there are now wrong.
h2. The configuration count and the Observe row
The section opens with "Three configurations matter throughout the operator"
and lists Default, Deploy and Observe. There is now a fourth source, the
runtime configuration, fetched from the cluster and cached separately from the
derived configurations.
The Observe row describes that configuration as derived purely from the last
reconciled spec. {{FlinkResourceContext#getObserveConfig}} now layers the
runtime configuration on top, so REST-observed values win over the spec.
h2. The warning is now false
The section closes with:
{quote}
None of these is the running configuration. The operator derives its view from
the spec and its own defaults, while the cluster can pick up settings the
operator never sees: a config.yaml baked into the image, environment overrides,
or properties the job sets programmatically. The configuration a pipeline
actually runs with can therefore differ from everything the operator tracks.
{quote}
Reading back settings that the job sets programmatically is exactly what
FLINK-35746 implemented, so this tells the reader the operator cannot do the
thing it now does. A narrower version of the caution is still true and worth
keeping, because the mapping covers only a subset of settings and only while
the job runs.
h2. The mechanism is undocumented
Nothing describes when the runtime configuration is fetched (once per job by
the job status observer, skipped for globally terminal jobs), how it is cached
(per resource and job id, under the existing config cache limits), which
settings it covers, or what happens when a fetch fails.
h1. Global job parameters are ingested unfiltered
{{FlinkRuntimeConfigurationUtils#mapJobConfiguration}} copies the job's global
parameters wholesale:
{code:java}
jobConfig.putAll(execInfo.getGlobalJobParameters());
{code}
Every other value produced by that class is an explicit, curated
{{ConfigOption}}. This one is unbounded and set by the job, and the result is
written over the observed configuration through {{observeConfig::setString}}.
A job that declares a global parameter matching an operator key, for example
under {{kubernetes.operator.}} or {{job.autoscaler.}}, therefore changes how
the operator manages that resource, with no indication in the logs. Keys in
those namespaces have no legitimate reason to arrive from a job's global
parameters, so they should be dropped.
h1. Expected outcome
The Configuration section describes the runtime configuration as a first-class
source, states that it overrides the observed configuration, and replaces the
warning with an accurate account of what the operator can and cannot observe.
Global job parameters in the operator and autoscaler namespaces are ignored
rather than applied, and the drop is logged so it is diagnosable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
