Hi Shaoxuan, Thanks for doing more efforts for the enhances of the scalability and the ease of use of Flink ML and make it one step further. Thank you for sharing a lot of context information.
big +1 for this proposal! Here only one suggestion, that is, It has been a short time until the release of flink-1.9, so I recommend It's better to add a detailed implementation plan to FLIP and google doc. What do you think? Best, Jincheng Shaoxuan Wang <wshaox...@gmail.com> 于2019年4月29日周一 上午10:34写道： > Hi everyone, > > Weihua has proposed to rebuild Flink ML pipeline on top of TableAPI several > months ago in this mail thread: > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Embracing-Table-API-in-Flink-ML-td25368.html > > Luogen, Becket, Xu, Weihua and I have been working on this proposal > offline in > the past a few months. Now we want to share the first phase of the entire > proposal with a FLIP. In this FLIP-39, we want to achieve several things > (and hope those can be accomplished and released in Flink-1.9): > > - > > Provide a new set of ML core interface (on top of Flink TableAPI) > - > > Provide a ML pipeline interface (on top of Flink TableAPI) > - > > Provide the interfaces for parameters management and pipeline/mode > persistence > - > > All the above interfaces should facilitate any new ML algorithm. We will > gradually add various standard ML algorithms on top of these new > proposed > interfaces to ensure their feasibility and scalability. > > > Part of this FLIP has been present in Flink Forward 2019 @ San Francisco by > Xu and Me. > > > https://sf-2019.flink-forward.org/conference-program#when-table-meets-ai--build-flink-ai-ecosystem-on-table-api > > > https://sf-2019.flink-forward.org/conference-program#high-performance-ml-library-based-on-flink > > You can find the videos & slides at > https://www.ververica.com/flink-forward-san-francisco-2019 > > The design document for FLIP-39 can be found here: > > > https://docs.google.com/document/d/1StObo1DLp8iiy0rbukx8kwAJb0BwDZrQrMWub3DzsEo > > > I am looking forward to your feedback. > > Regards, > > Shaoxuan >
