Re: Explanation on limitations of the Flink Table API

2016-04-21 Thread Simone Robutti
Thanks for all your input. The design document covers the use cases we have in mind and querying external sources may be interesting to us for other uses not mentioned in the first mail. I will wait for developments in this direction, because the expected result seems promising. :) Thank you aga

Re: Explanation on limitations of the Flink Table API

2016-04-21 Thread Flavio Pompermaier
We're also trying to work around the current limitations of Table API and we're reading DataSets with on-purpose input formats that returns a POJO Row containing the list of values (but we're reading all values as String...). Actually we would also need a way to abstract the composition of Flink op

Re: Explanation on limitations of the Flink Table API

2016-04-21 Thread Fabian Hueske
Hi Simone, in Flink 1.0.x, the Table API does not support reading external data, i.e., it is not possible to read a CSV file directly from the Table API. Tables can only be created from DataSet or DataStream which means that the data is already converted into "Flink types". However, the Table API

Explanation on limitations of the Flink Table API

2016-04-21 Thread Simone Robutti
Hello, I would like to know if it's possible to create a Flink Table from an arbitrary CSV (or any other form of tabular data) without doing type safe parsing with expliciteky type classes/POJOs. To my knowledge this is not possible but I would like to know if I'm missing something. My requiremen