Re: Limitations with Retract Streams on SQL

2018-05-23 Thread Fabian Hueske
Hi Gregory, Rong's analysis is correct. The UNION with duplicate elimination is translated into a UNION ALL and a subsequent grouping operator on all attributes without an aggregation function. Flink assumes that all grouping operators can produce retractions (updates) and window-grouped aggregate

Re: Limitations with Retract Streams on SQL

2018-05-22 Thread Rong Rong
The SQL UNION is the reason here that's causing (a) the table is not append only, and (b) the inner GroupBy. If you check out the UNION operator[1], it suggests that: "Any duplicate records are automatically removed unless UNION ALL is used". So: (1) it is definitely not append-only operation as y