Hi Surendra! The resource configuration in Flink is pretty well established and covers setting both memory requests and limits (through the limit factor.)
Could you please elaborate why you think this change is a good addition? I see a few downsides: - It complicates memory configuration by adding new options without actually enabling anything new - Existing pipelines with podTemplates may suddenly start running with different memory settings in prod after this change So at this point I am slightly against making this change, but I would like to hear the thoughts of the community on this matter. Thanks! Gyula On Thu, Nov 23, 2023 at 2:00 PM Surendra Singh Lilhore < surendralilh...@apache.org> wrote: > Hello everyone, > > I've encountered an issue while using the flink open source > kubernetes operator for Flink deployment. Despite setting resource limits > in the pod template, it appears that these limits are not considered during > TaskManager (TM) pod deployment. Upon code investigation, it seems the > limits are being overridden by the default limit factor in > KubernetesUtils#getResourceRequirements() > < > https://github.com/apache/flink/blob/master/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/utils/KubernetesUtils.java#L372 > > > . > > The current behavior of Flink only considers the limit from the default > factor, neglecting pod template resource limits. I propose Flink should > incorporate both the limit factor and pod template resource limits, taking > the maximum value. > > I've raised the issue and submitted a pull request: FLINK-33609 > <https://github.com/apache/flink/pull/23768> > > During the review process, a valid concern was raised regarding the > proposed changes. The suggestion is to initiate a quick discussion, as this > modification will significantly alter the resource handling logic. It's > emphasized that maintaining consistency in the logic for both resource > requests and limits is crucial, rather than applying changes to only one of > them. > > I would appreciate any feedback on this. > > Thank you for your time and contributions to the Flink project. > > Thank you, > Surendra >
