I didn't know that paper... Thanks for sharing. I've worked on a SQL layer for Stratosphere some time ago, using Apache Calcite (called Optiq back then). I think the project provides a lot of very good tooling for creating a SQL layer. So if we decide to go for SQL on Flink, I would suggest to use Calcite. I can also help you a bit with Calcite to get started with it.
I agree with Fabian that it would probably make more sense for now to enhance the Table API. I think the biggest limitation right now is that it only supports POJOs. We should also support Tuples (I know thats difficult to do), data from HCatalog (that includes parquet & orc), JSON, ... Then, I would add filter and projection pushdown into the table API. On Tue, May 26, 2015 at 10:03 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: > It would also be relatively simple (I think) to retarget drill to Flink if > Flink doesn't provide enough typing meta-data to do traditional SQL. > > > > On Tue, May 26, 2015 at 12:52 PM, Fabian Hueske <fhue...@gmail.com> wrote: > > > Hi, > > > > Flink's Table API is pretty close to what SQL provides. IMO, the best > > approach would be to leverage that and build a SQL parser (maybe together > > with a logical optimizer) on top of the Table API. Parser (and optimizer) > > could be built using Apache Calcite which is providing exactly this. > > > > Since the Table API is still a fairly new component and not very feature > > rich, it might make sense to extend and strengthen it before putting > > something major on top. > > > > Cheers, Fabian > > > > 2015-05-26 21:38 GMT+02:00 Timo Walther <twal...@apache.org>: > > > > > Hey everyone, > > > > > > I would be interested in having a complete SQL API in Flink. How is the > > > status there? Is someone already working on it? If not, I would like to > > > work on it. I found http://ijcsi.org/papers/IJCSI-12-1-1-169-174.pdf > but > > > I couldn't find anything on the mailing list or Jira. Otherwise I would > > > open an issue and start a discussion about it there. > > > > > > Regards, > > > Timo > > > > > >
