Interaction of watermarks and windows

2020-06-21 Thread Sergii Mikhtoniuk
Greetings, When playing around with the following simple event-time stream aggregation: SELECT TUMBLE_START(event_time, INTERVAL '1' DAY) as event_time, ... FROM input GROUP BY TUMBLE(event_time, INTERVAL '1' DAY), symbol ...to my surprise I found out that the t

Re: Process available data and stop with savepoint

2020-05-18 Thread Sergii Mikhtoniuk
case. > > ------ > *From:* Arvid Heise > *Sent:* Monday, May 18, 2020 11:15:09 PM > *To:* Congxian Qiu > *Cc:* Sergii Mikhtoniuk ; user < > user@flink.apache.org> > *Subject:* Re: Process available data and stop with savepoint > > Hi Se

Process available data and stop with savepoint

2020-05-17 Thread Sergii Mikhtoniuk
Hello, I'm migrating my Spark-based stream processing application to Flink (Calcite SQL and temporal tables look too attractive to resist). My Spark app works as follows: - application is started periodically - it reads a directory of Parquet files as a stream - SQL transformations are applied -