Hi Xianda, thanks for sharing this detailed feedback. Do I understand you correctly that flink-python and flink-streaming-python are not usable for the use cases at Alibaba atm?
Could you share a bit more details about the Python UDFs for SQL? How do you execute the Python code? Will it work with any Python library? If you are about to publish the design document then feel free to refer me to this document. Cheers, Till On Mon, Dec 10, 2018 at 3:08 AM Xianda Ke <kexia...@gmail.com> wrote: > Xianda Ke <kexia...@gmail.com> > 9:47 AM (11 minutes ago) > to dev, user > After communicating with some of the internal users at Alibaba, my > impression is that: > > - Most of them need C extensions support, they want to integrated their > algorithms with stream processingļ¼but Jython is unacceptable for them. > - For some users, who are only familiar with SQL/Python, developing Java > API application/UDF is too complex. Writing Python UDF and declaring it > in > SQL is preferred. > - Machine Learning users needs richer Python APIs, such as Table API > Python support. > > > From my point of view, currently Python support has a few caveats in Flink. > > - For batch, there is only DataSet Python API. > - For streaming, where Flink really shines, only Jython is supported, > but Jython has lots of limitations. > - For most of the big data users, SQL/Table API is more friendly, but > Python users have no such APIs right now. > - The interactive Python shell is very user-friendly. It can be used to > test interactively and is a simple way to learn the API. However, there > is > no such interactive Python shell in Flink now. > > > At Alibaba, Python UDF for SQL has been developed and has been delivered to > internal users. Currently, we start to develop the Python API, and we've > drafted a design documentation and will publish it to the community soon > for discussion. > > Regards, > Xianda > > On Fri, Dec 7, 2018 at 11:30 PM Till Rohrmann <trohrm...@apache.org> > wrote: > > > Dear Flink community, > > > > in order to better understand the needs of our users and to plan for the > > future, I wanted to reach out to you and ask how much you use Flink's > > Python API, namely flink-python and flink-streaming-python. > > > > In order to gather feedback, I would like to ask all Python users to > > respond to this thread and quickly outline how you use Python in > > combination with Flink. Thanks a lot for your help! > > > > Cheers, > > Till > > > > > -- > Ke, Xianda >
