After communicating with some of the internal users at Alibaba, my impression is that: * Most of them need C extentions support, they want to integrated their algorithms with stream processingļ¼but Jython is unacceptable for them. * For some users, who are only familiar with SQL/Python, developing Java API application/UDF is too complex. Writing Python UDF and declaring it in SQL is preferred. * Machine Learning users needs richer Python APIs, such as Table API Python support.
From my point of view, currently Python support has a few caveats in Flink. * For batch, there is only DataSet Python API. * For streaming, where Flink really shines, only Jython is supported, but Jython has lots of limitations. * For most of the big data users, SQL/Table API is more friendly, but Python users have no such APIs right now. * The interactive Python shell is very user-friendly. It can be used to test interactively and is a simple way to learn the API. However, there is no such interactive Python shell in Flink now. At Alibaba, Python UDF for SQL has been developed and has been delivered to internal users. Currently, we start to develop the Python API, and we've drafted a design documentation and will publish it to the community soon for discussion. Regards, Xianda > On Dec 7, 2018, at 11:29 PM, Till Rohrmann <trohrm...@apache.org> wrote: > > Dear Flink community, > > in order to better understand the needs of our users and to plan for the > future, I wanted to reach out to you and ask how much you use Flink's > Python API, namely flink-python and flink-streaming-python. > > In order to gather feedback, I would like to ask all Python users to > respond to this thread and quickly outline how you use Python in > combination with Flink. Thanks a lot for your help! > > Cheers, > Till
