Hi all,

We run hundreds of Flink pipelines across several shared GKE clusters and
experience significant cluster overhead. By cluster overhead, we mean
allocatable CPU and memory available on active nodes that remain
unrequested but cannot be reclaimed because the nodes still host pods.


This comes mainly from two sources:


   -

   TaskManager CPU/memory request ratios do not always align with our
   machine shapes, leaving capacity stranded when either CPU or memory is
   exhausted first.
   -

   Deployments and autoscaling gradually scatter TaskManagers across nodes.
   Because TaskManagers are bare pods, GKE cannot evict them for
   consolidation, and Kubernetes does not proactively repack them.


As a result, partially occupied nodes can remain running indefinitely
despite substantial aggregate free capacity.

Have others operating large shared Flink clusters encountered this? How do
you manage resource-shape mismatch and fragmentation?


Thanks,

Yanis

Reply via email to