Hi Haohui, the Flink community started about a year ago to port its Table API on top of Apache Calcite and finalized the integration in June. Now, Calcite is the central component of Flink's relational APIs. Flink features SQL and the Table API, a language-integrated query (LINQ) API. Both APIs are translated into a common logical representation, optimized by Calcite and executed on Flink's DataStream or DataSet APIs depending on stream or batch mode. The translation of the Table API and SQL and how Calcite is used is discussed in this blog post [1] and this presentation [2].
The streaming SQL part is currently limited to simple selections, projections, and table function joins. The Table API features grouped window aggregates in addition. We have written a design document for our future Stream SQL efforts [3]. The design is based on the concept of dynamic tables which are derived from streams using the stream-table duality. Dynamic tables are queried with regular (batch) SQL and can be converted back into streams or persisted as materialized views (either internally or in external kv-stores like Cassandra or HBase). The Table and SQL APIs have received a lot of contributions from many different people and are very actively developed. The Flink community is happy about everybody who wants to contribute and extend and improve Flink's relational API. Please let me know if you have any questions. Cheers, Fabian [1] http://flink.apache.org/news/2016/05/24/stream-sql.html [2] http://www.slideshare.net/FlinkForward/fabian-hueske-taking-a-look-under-the-hood-of-apache-flinks-relational-apis [3] https://docs.google.com/document/d/1qVVt_16kdaZQ8RTfA_f4konQPW4tnl8THw6rzGUdaqU 2016-12-20 5:27 GMT+01:00 Haohui Mai <ricet...@gmail.com>: > Hi, > > We are currently exploring on building a streaming SQL solution on top of > Flink. We think that Flink is a good fit for the problem. > > I'm curious about what is the status on the Flink-Calcite integration right > now? Is it being actively developed and on the road map? > > We are also very open to contribute back if it aligns with the interests > from the community. > > Your answers are appreciated. > > Thanks, > Haohui >
