HDFS streaming source concerns

2022-04-06 Thread Carlos Downey
Hi, We have an in-house platform that we want to integrate with external clients via HDFS. They have lots of existing files and they continuously put more data to HDFS. Ideally, we would like to have a Flink job that takes care of ingesting data as one of the requirements is to execute SQL on top

JVM cluster not firing event time window

2021-11-09 Thread Carlos Downey
Hello, Recently, I've decided to migrate one of my toy projects to Flink 1.14.0 and I realized that JVM cluster behaviour changed. It no longer respects event time. On Flink 1.12.5 I didn't experience any issues. I tried some debugging and it seems that InternalTimerServiceImpl's watermark is not