Hi everybody, the following design document shows different ways to execute Table API and SQL queries (query in the following) in Java and Scala programs. It proposes alternatives for data sources and result sinks of queries.
Data Sources - Use a DataSet, DataStream, or Table as input for a query - Use an external table (CSV, Parquet, ORC, Hbase, …) as input for a query - Register external catalogs and use their tables as input for a query Result Sinks - Emit the result of a query as DataSet, DataStream, or Table - Emit the result to an external storage system (file, JDBC, Kafka, …) These options allow to embed a query into a regular DataSet / DataStream program, as well as, specifying it without touching the DataSet or DataStream API at all by reading from external tables and writing to an external system. At the moment, Table API queries can only use DataSets as input and emit their results as DataSets. SQL queries are not supported yet. DESIGN DOCUMENT: --> https://docs.google.com/document/d/1sITIShmJMGegzAjGqFuwiN_iw1urwykKsLiacokxSw0/edit?usp=sharing Feedback and comments are welcome. Cheers, Fabian
