Hi All, As Xianda brought up in the previous email, There are a large number of data analysis users who want flink to support Python. At the Flink API level, we have DataStreamAPI/DataSetAPI/TableAPI&SQL, the Table API will become the first-class citizen. Table API is declarative and can be automatically optimized, which is mentioned in the Flink mid-term roadmap by Stephan. So we first considering supporting Python at the Table level to cater to the current large number of analytics users. For further promote Python support in flink table level. Dian, Wei and I discussed offline a bit and came up with an initial features outline as follows:
- Python TableAPI Interface Introduce a set of Python Table API interfaces, including interface definitions such as Table, TableEnvironment, TableConfig, etc. - Implementation Architecture We will offer two alternative architecture options, one for pure Python language support and one for extended multi-language design. - Job Submission Provide a way that can submit(local/remote) Python Table API jobs. - Python Shell Python Shell is to provide an interactive way for users to write and execute flink Python Table API jobs. The design document for FLIP-38 can be found here: https://docs.google.com/document/d/1ybYt-0xWRMa1Yf5VsuqGRtOfJBz4p74ZmDxZYg3j_h8/edit?usp=sharing I am looking forward to your comments and feedback. Best, Jincheng
