Re: Blink Planner Retracting Streams

2020-06-18 Thread John Mathews
Below is a basic unit test of what we are trying to achieve, but basically, we are trying to convert from a retracting stream to a RetractingStreamTableSink, which is easily done with the CRow from the original flink planner, but seems to be very difficult to do with the blink planner. The below f

Re: Blink Planner Retracting Streams

2020-06-18 Thread John Mathews
So the difference between Tuple2 and CRow is that CRow has a special TypeInformation defined here: https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/runtime/types/CRowTypeInfo.scala#L32 that returns the TypeInfo of the underlying row

Re: Blink Planner Retracting Streams

2020-06-17 Thread Jark Wu
Hi John, Maybe I misunderstand something, but CRow doesn't have the `getSchema()` method. You can getSchema() on the Table, this also works if you convert the table into Tuple2. Actually, there is no big difference between CRow and Tuple2, they both wrap the change flag and the Row. Best, Jark

Re: Blink Planner Retracting Streams

2020-06-17 Thread John Mathews
Hello Godfrey, Thanks for the response! I think the problem with Tuple2, is that if my understanding is correct of how CRow worked, when CRow's getSchema() was returned it would return the underlying schema of the row it contained. Tuple2 doesn't do that, so it changes/breaks a lot of our downstr

Re: Blink Planner Retracting Streams

2020-06-16 Thread godfrey he
hi John, You can use Tuple2[Boolean, Row] to replace CRow, the StreamTableEnvironment#toRetractStream method return DataStream[(Boolean, T)]. the code looks like: tEnv.toRetractStream[Row](table).map(new MapFunction[(Boolean, Row), R] { override def map(value: (Boolean, Row)): R = ...

Blink Planner Retracting Streams

2020-06-16 Thread John Mathews
Hello, I am working on migrating from the flink table-planner to the new blink one, and one problem I am running into is that it doesn't seem like Blink has a concept of a CRow, unlike the original table-planner. I am therefore struggling to figure out how to properly convert a retracting stream