Hi Wencong, Thanks for your clarification! +1
Best regards, Jing On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu <liuwencle...@163.com> wrote: > Hi Jing, > > > Thanks for your reply！ > > > > Or the "independent module extraction" mentioned in the FLIP does mean an > independent module in Flink? > > > Yes. If there are submodules in Flink repository needs the iteration > (currently not), > we could consider extracting them to a new submodule of Flink. > > > > users will have to add one more dependency of Flink ML. If iteration is > the > only feature they need, it will look a little bit weird. > > > If users only need to execute iteration jobs, they can simply remove the > Flink > dependency and add the necessary dependencies related to Flink ML. > However, > they can still utilize the DataStream API as it is also a dependency of > Flink ML. > > > Keeping an iteration submodule in Flink repository and make Flink ML > depends it > is also another solution. But the current implementation of Iteration in > DataStream > should be removed definitely due to its Incompleteness. > > > The placement of the Iteration API in the repository is a topic that has > multiple > potential solutions. WDYT? > > > Best, > Wencong > > > > > > > > > > > > At 2023-09-01 17:59:34, "Jing Ge" <j...@ververica.com.INVALID> wrote: > >Hi Wencong, > > > >Thanks for the proposal! > > > >"The Iteration API in DataStream is planned be deprecated in Flink 1.19 > and > >then finally removed in Flink 2.0. For the users that rely on the > Iteration > >API in DataStream, they will have to migrate to Flink ML." > >- Does it make sense to migrate the iteration module into Flink directly? > >Or the "independent module extraction" mentioned in the FLIP does mean an > >independent module in Flink? Since the iteration will be removed in Flink, > >users will have to add one more dependency of Flink ML. If iteration is > the > >only feature they need, it will look a little bit weird. > > > > > >Best regards, > >Jing > > > >On Fri, Sep 1, 2023 at 11:05 AM weijie guo <guoweijieres...@gmail.com> > >wrote: > > > >> Thanks, +1 for this. > >> > >> Best regards, > >> > >> Weijie > >> > >> > >> Yangze Guo <karma...@gmail.com> 于2023年9月1日周五 14:29写道： > >> > >> > +1 > >> > > >> > Thanks for driving this. > >> > > >> > Best, > >> > Yangze Guo > >> > > >> > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song <tonysong...@gmail.com> > >> wrote: > >> > > > >> > > +1 > >> > > > >> > > Best, > >> > > > >> > > Xintong > >> > > > >> > > > >> > > > >> > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin <lindon...@gmail.com> > wrote: > >> > > > >> > > > Thanks Wencong for initiating the discussion. > >> > > > > >> > > > +1 for the proposal. > >> > > > > >> > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu <liuwencle...@163.com > > > >> > wrote: > >> > > > > >> > > > > Hi devs, > >> > > > > > >> > > > > I would like to start a discussion on FLIP-357: Deprecate > Iteration > >> > API > >> > > > of > >> > > > > DataStream [1]. > >> > > > > > >> > > > > Currently, the Iteration API of DataStream is incomplete. For > >> > instance, > >> > > > it > >> > > > > lacks support > >> > > > > for iteration in sync mode and exactly once semantics. > >> Additionally, > >> > it > >> > > > > does not offer the > >> > > > > ability to set iteration termination conditions. As a result, > it's > >> > hard > >> > > > > for developers to > >> > > > > build an iteration pipeline by DataStream in the practical > >> > applications > >> > > > > such as machine learning. > >> > > > > > >> > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has > >> introduced > >> > a > >> > > > > unified iteration library > >> > > > > in the Flink ML repository. This library addresses all the > issues > >> > present > >> > > > > in the Iteration API of > >> > > > > DataStream and could provide solution for all the iteration > >> > use-cases. > >> > > > > However, maintaining two > >> > > > > separate implementations of iteration in both the Flink > repository > >> > and > >> > > > the > >> > > > > Flink ML repository > >> > > > > would introduce unnecessary complexity and make it difficult to > >> > maintain > >> > > > > the Iteration API. > >> > > > > > >> > > > > As such I propose deprecating the Iteration API of DataStream > and > >> > > > removing > >> > > > > it completely in the next > >> > > > > major version. In the future, if other modules in the Flink > >> > repository > >> > > > > require the use of the > >> > > > > Iteration API, we can consider extracting all Iteration > >> > implementations > >> > > > > from the Flink ML repository > >> > > > > into an independent module. > >> > > > > > >> > > > > Looking forward to your feedback. > >> > > > > > >> > > > > > >> > > > > [1] > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream > >> > > > > [2] > >> > > > > > >> > > > > >> > > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300 > >> > > > > > >> > > > > Best regards, > >> > > > > > >> > > > > Wencong Liu > >> > > > > >> > > >> >
