Hi, Thanks for driving this FLIP, +1 for the proposed changes.
Limit the maximum value of shuffle read memory is very useful when using when using adaptive batch scheduler. Currently, the adaptive batch scheduler may cause a large number of input channels in a certain TM, so we generally recommend that users configure "taskmanager.network.memory.buffers-per-channel: 0" to decrease the the possibility of “Insufficient number of network buffers” error. After this FLIP, users no longer need to configure the "taskmanager.network.memory.buffers-per-channel". So +1 from my side. Best, Lijie Xintong Song <tonysong...@gmail.com> 于2022年12月20日周二 10:04写道： > Thanks for the proposal, Yuxin. > > +1 for the proposed changes. I think these are indeed helpful usability > improvements. > > Best, > > Xintong > > > > On Mon, Dec 19, 2022 at 3:36 PM Yuxin Tan <tanyuxinw...@gmail.com> wrote: > > > Hi, devs, > > > > I'd like to start a discussion about FLIP-266: Simplify network memory > > configurations for TaskManager[1]. > > > > When using Flink, users may encounter the following issues that affect > > usability. > > 1. The job may fail with an "Insufficient number of network buffers" > > exception. > > 2. Flink network memory size adjustment is complex. > > When encountering these issues, users can solve some problems by adding > or > > adjusting parameters. However, multiple memory config options should be > > changed. The config option adjustment requires understanding the detailed > > internal implementation, which is impractical for most users. > > > > To simplify network memory configurations for TaskManager and improve > Flink > > usability, this FLIP proposed some optimization solutions for the issues. > > > > Looking forward to your feedback. > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-266%3A+Simplify+network+memory+configurations+for+TaskManager > > > > Best regards, > > Yuxin > > >
