Dennis-Mircea Ciupitu created FLINK-40414:
---------------------------------------------
Summary: Autoscaler state ConfigMap is treated as trusted input
Key: FLINK-40414
URL: https://issues.apache.org/jira/browse/FLINK-40414
Project: Flink
Issue Type: Bug
Components: Autoscaler, Kubernetes Operator
Reporter: Dennis-Mircea Ciupitu
h1. Summary
The autoscaler persists its state in a per-resource ConfigMap
({{autoscaler-<name>}}): scaling history, collected metrics, scaling tracking,
delayed scale-down, parallelism overrides and config overrides. The operator
reads that ConfigMap back on every cycle and acts on its contents, but treats
it as trusted input. The ConfigMap is writable by any workload in the
namespace, so a poisoned or corrupt entry can break a reconcile or influence a
managed deployment.
h1. Trust boundary
The autoscaler state ConfigMap is created and owned by the operator, but the
default Flink job Role grants the job service account
{{create/update/patch/delete}} on {{configmaps}} in its namespace (required for
Kubernetes HA). Tenant code running in a TaskManager or JobManager pod can
therefore write any ConfigMap in the namespace, including the autoscaler state
of another job.
Among the ConfigMaps the operator handles, only the autoscaler state ConfigMap
has its content deserialized and acted upon. The generated cluster config and
pod-template ConfigMaps are write-only from the operator, and the Kubernetes HA
ConfigMaps are read only by name and key presence, never by value. The
autoscaler state ConfigMap is the operator's untrusted-input surface.
h1. Problems
h2. Reads are not uniformly fail-closed
The six read paths handle a bad stored value inconsistently:
* {{getParallelismOverrides}} has no error handling. A malformed value throws
out of the read and into the reconcile loop.
* {{getConfigChanges}} falls back to empty on a bad value but does not discard
the poisoned entry, so it is re-read and re-parsed every cycle.
* The remaining four discard the bad entry and fall back to empty.
So the same class of input is variously ignored, re-read, or allowed to
propagate an exception.
h2. Config overrides are applied to the deployment spec unfiltered
When memory tuning is enabled, the config overrides read from the ConfigMap are
merged into {{spec.flinkConfiguration}} with no key filtering. A poisoned entry
can therefore inject arbitrary Flink configuration, for example
{{env.java.opts}} or a pod template, into a managed deployment. This is an
escalation from "write one ConfigMap" to influencing how and what the managed
cluster runs, potentially across jobs within a namespace.
h2. No value sanity
Values read from the ConfigMap, such as the per-vertex parallelism overrides,
are applied without basic validity checks.
h1. Impact
* Integrity and availability: a crafted entry can break a reconcile through an
unhandled exception, or feed the autoscaler bogus history and metrics that
steer scaling for that job or a co-tenant.
* Config injection: with memory tuning enabled, arbitrary Flink configuration
reaches the managed deployment spec.
* Current exposure is bounded by {{job.autoscaler.memory.tuning.enabled}}
(default off) for the injection path, but the design flaw is independent of
that flag.
This is not a deserialization remote-code-execution issue. The mapper uses no
default typing and binds to concrete types, so malformed input fails to bind
rather than instantiating arbitrary classes. The risk is integrity,
availability and config injection.
h1. Scope
This completes the autoscaler-adjacent security review alongside FLINK-40400
(artifact fetch SSRF) and FLINK-40401 (decompression bomb on the compressed
state fields). Those two harden individual fields or paths as issues surfaced.
This item addresses the general untrusted-read boundary of the state ConfigMap
so the class of problem is closed rather than one instance.
h1. Proposed approach
At a high level:
* Treat every read from the autoscaler state ConfigMap as untrusted:
deserialize uniformly and fail closed, discarding the offending entry and
falling back to empty on any failure, plus light value sanity on the values
that are applied.
* Filter config overrides on apply against the set of keys memory tuning is
actually allowed to set, so that only memory-tuning keys can reach the
deployment spec.
Out of scope: deeper hardening of scaling decisions against plausible but wrong
metrics, and the decompression size bound owned by FLINK-40401.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
