Thanks Zhanghao for driving this FLIP. +1 for this.
Best, Weihua On Mon, Nov 20, 2023 at 5:49 PM Zhanghao Chen <zhanghao.c...@outlook.com> wrote: > Hi all, > > I'd like to start a discussion of FLIP-395: Deprecate Global Aggregator > Manager [1]. > > Global Aggregate Manager was introduced in [2] to support event time > synchronization across sources and more generally, coordination of parallel > tasks. AFAIK, this was only used in the Kinesis source for an early version > of watermark alignment. Operator Coordinator, introduced in FLIP-27, > provides a more powerful and elegant solution for that need and is part of > the new source API standard. FLIP-217 further provides a complete solution > for watermark alignment of source splits on top of the Operator Coordinator > mechanism. Furthermore, Global Aggregate Manager manages state in JobMaster > object, causing problems for adaptive parallelism changes [3]. > > Therefore, I propose to deprecate the use of Global Aggregate Manager, > which can improve the maintainability of the Flink codebase without > compromising its functionality. > > Looking forward to your feedbacks, thanks. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-395%3A+Deprecate+Global+Aggregator+Manager > [2] https://issues.apache.org/jira/browse/FLINK-10886 > [3] https://issues.apache.org/jira/browse/FLINK-31245 > > Best, > Zhanghao Chen >
