Re: Idle source configuration per topic with the Kafka Table API connector

2021-05-31 Thread Svend
Awesome, thanks a lot for clarifications Jing Zhang, it's very useful. Best, Svend On Sun, 30 May 2021, at 6:27 AM, JING ZHANG wrote: > Hi Svend, > Your solution could work well in Flink 1.13.0 and Flink 1.13.0+ because those > version provides many related improvements. > > > as per [1] > Ye

Re: Idle source configuration per topic with the Kafka Table API connector

2021-05-29 Thread JING ZHANG
Hi Svend, Your solution could work well in Flink 1.13.0 and Flink 1.13.0+ because those version provides many related improvements. > as per [1] Yes, "table.exec.source.idle-timeout" is not table-level parameter, but a global parameter, It would apply to all those table sources which with watermar

Idle source configuration per topic with the Kafka Table API connector

2021-05-29 Thread Svend
Hi everyone, My Flink streaming application consumes several Kafka topics, one of which receiving traffic in burst once per day. I would like that topic not to hold back the progress of the watermark. Most of my code is currently using the SQL API and in particular the Table API Kafka connecto