Re: Table API and registration of DataSet/DataStream

2017-09-14 Thread Flavio Pompermaier
I see...anyway for me it continue to be very misleading to have different syntax for where clauses (SQL vs scala)... Why not make them compatible? Is it that complex? On Thu, Sep 14, 2017 at 4:26 PM, Fabian Hueske wrote: > Hi Flavio, > > 1) The Java Table API does not aim to resemble SQL but th

Re: Table API and registration of DataSet/DataStream

2017-09-14 Thread Fabian Hueske
Hi Flavio, 1) The Java Table API does not aim to resemble SQL but the Scala Table API which is integrated with the host language (Scala). Hence the different syntax for expressions. 2) Yes, that would be one way to do it. If that adds to much boilerplate code, you could encapsulate the code in yo

Re: Table API and registration of DataSet/DataStream

2017-09-14 Thread Flavio Pompermaier
Hi Fabian, basically these were my problems with Table API. 1 ) Table.sql() has a different where syntax than Table.where() , and this is very annoying (IMHO). Ex: Table.sql("SELECT * FROM XXX WHERE Y IS NOT NULL) vs Table.i.where("Y.isNotNull"). 2) If I understood correctly, my program that id

Re: Table API and registration of DataSet/DataStream

2017-09-14 Thread Fabian Hueske
Not sure what you mean by "translate a where clause to a filter function". Isn't that exactly what Table.filter(String condition) is doing? It translates a SQL-like condition (represented as String) into an operator that filter the Table. 2017-09-09 23:49 GMT+02:00 Flavio Pompermaier : > Yes I

Re: Table API and registration of DataSet/DataStream

2017-09-09 Thread Flavio Pompermaier
Yes I can do that of course. What I need is basically the possibility to translate a where clause to a filter function. Is there any utility class that does that in Flink? On 9 Sep 2017 21:54, "Fabian Hueske" wrote: > Hi Flavio, > > I tried to follow your example. If I got it right, you would li

Re: Table API and registration of DataSet/DataStream

2017-09-09 Thread Fabian Hueske
Hi Flavio, I tried to follow your example. If I got it right, you would like to change the registered table by assigning a different DataStream to the original myDs variable. With registerDataStream("test", myDs, ...) you don't register the variable myDs as a table but it's current value, i.e., a

Table API and registration of DataSet/DataStream

2017-09-08 Thread Flavio Pompermaier
Hi to all, I have a doubt about Table API. Let's say my code is something like: StreamTableEnvironment te = ...; RowTypeInfo rtf = new RowTypeInfo(...); DataStream myDs = te.registerDataStream("test",myDs,columnNames); Table table = te.sql("SELECT *, (NAME = 'John') as VALID FROM test WHERE ..."