Hi Arvid and Jiangjie, thanks to both of you for the quick and valuable response. I will take a look at the linked projects.
Kind Regards, Niklas -- niklas.wil...@uniberg.com Mobile: +49 160 9793 2593 Office: +49 40 2380 6523 Simon-von-Utrecht-Straße 85a 20359 Hamburg UNIBERG GmbH Registergericht: Amtsgericht Kiel HRB SE-1507 Geschäftsführer: Andreas Möller, Martin Ulbricht Information Art. 13 DSGVO B2B: Für die Kommunikation mit Ihnen verarbeiten wir ggf. Ihre personenbezogenen Daten. Alle Informationen zum Umgang mit Ihren Daten finden Sie unter https://www.uniberg.com/impressum.html. > On 20. Nov 2020, at 03:07, Becket Qin <becket....@gmail.com> wrote: > > Hi Niklas, > > We dropped the Flink ML lib in 1.9 and plan to replace it with a new machine > learning library for traditional machine learning algorithms. And that > library will be based on FLIP-39. The plan was pushed back a little bit > because we plan to deprecate DataSet API and but haven't got the batch > iteration support in DataStream API yet. So at this point we don't have an ML > lib implementation in Flink. > > That being said, we are working with the community to add some ML related > features to Flink. At this point, we have the following two projects > available from Alibaba that will likely be contributed to Flink. You may also > take a look at them. > > Alink - A machine learning library. > https://github.com/alibaba/alink <https://github.com/alibaba/alink> > > Flink-AI-Extended - A project helps running TF / PyTorch on top of Flink. > https://github.com/alibaba/flink-ai-extended > <https://github.com/alibaba/flink-ai-extended> > > Thanks, > > Jiangjie (Becket) Qin > > On Fri, Nov 20, 2020 at 3:43 AM Arvid Heise <ar...@ververica.com > <mailto:ar...@ververica.com>> wrote: > Hi Niklas, > > indeed some efforts on the machine learning libraries are pushed back in > favor of getting proper PyTorch and Tensorflow support through PyFlink. > > Native implementations in Flink have been done so far in the DataSet API, > which is going to deprecated in the next few releases in favor of the unified > DataStream API with bounded streams. I expect efforts for native > implementations to be picked up once DataSet is fully replaced to avoid > doubling the work. One of the most important features that is lacking is > proper iteration support in DataStream. > > On Thu, Nov 19, 2020 at 1:34 PM Niklas Wilcke <niklas.wil...@uniberg.com > <mailto:niklas.wil...@uniberg.com>> wrote: > Hi Flink-Community, > > I'm digging through the history of FlinkML and FLIP-39 [0]. What I understood > so far is that FlinkML has been removed in 1.9, because it got unmaintained. > I'm not really able to find out whether FLIP-39 and providing a replacement > for FlinkML is currently worked on. The Umbrella Jira Ticket FLINK-12470 [1] > looks stale to me. > Was there maybe a change of strategy in the meantime? Is the focus currently > on PyFlink to provide ML-Solutions (FLIP-96 [2])? > It would be really interesting to get some insights about the future and > roadmap of ML in the Flink ecosystem. Thank you very much! > > Kind Regards, > Niklas > > [0] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs> > [1] https://issues.apache.org/jira/browse/FLINK-12470 > <https://issues.apache.org/jira/browse/FLINK-12470> > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-96%3A+Support+Python+ML+Pipeline+API > > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-96%3A+Support+Python+ML+Pipeline+API> > > -- > Arvid Heise | Senior Java Developer > <https://www.ververica.com/> > Follow us @VervericaData > -- > Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference > Stream Processing | Event Driven | Real Time > -- > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > -- > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > (Toni) Cheng
smime.p7s
Description: S/MIME cryptographic signature
