Re: Flink Table API watermark after a select operation on a table

2023-06-25 Thread feng xiangyu
Hi Eugenio, According to docs[1], there are two ways to define the watermark in a table: 1. Defining in DDL 2. During DataStream-to-Table Conversion In your case, I think could use CREATE TABLE DDL to create a new table from filteredPhasesDurationsTable with watermark. See more in CREATE Statemen

Re: Flink Table API watermark after a select operation on a table

2023-06-25 Thread feng xiangyu
Hi, Eugenio AFAIK, you could define watermark on the data_fine by adding attribute in phasesDurationsSchema. For example: final Schema phasesDurationsSchema = Schema.newBuilder() .column("id_fascicolo", DataTypes.BIGINT().notNull()) .column("nrg", DataTypes.STRING()) .column("giudice", DataTypes.

Flink Table API watermark after a select operation on a table

2023-06-25 Thread Eugenio Marotti
Hi everyone, I'm using Flink for processing some streaming data. First of all I have two tables receiving events from Kafka. These tables are joined and the resulting table is converted to a DataStream where it is processed by a custom KeyedProcessFunction. The output is then converted to a tab

Re: table.exec.state.ttl not working as expected

2023-06-25 Thread Jane Chan
Hi, >From the attachment, I saw there are IntervalJoin and GroupWindowAggregate operators. AFAIK the state retention for such operators is not controlled by `table.exec.state.ttl`. Could you share the operator-level state metrics to help identify the issue? Best, Jane On Sun, Jun 25, 2023 at 10: