Re: [DISCUSS] FLIP-144: Native Kubernetes HA for Flink

Hi Till, thanks for your valuable feedback. 1. Yes, leader election and storing leader information will use a same ConfigMap. When a contender successfully performs a versioned annotation update operation to the ConfigMap, it means that it has been elected as the leader. And it will write the lead

Streaming SQL Job Switches to FINISHED before all records processed

Hey all, I'm not sure if I've missed something in the docs, but I'm having a bit of trouble with a streaming SQL job that starts w/ raw SQL queries and then transitions to a more traditional streaming job. I'm on Flink 1.10 using the Blink planner, running locally with no checkpointing. The job l

Re: Checkpoint metadata deleted by Flink after ZK connection issues

Hi all, I will have a look. Kostas On Mon, Sep 28, 2020 at 3:56 PM Till Rohrmann wrote: > > Hi Cristian, > > thanks for reporting this issue. It looks indeed like a very critical problem. > > The problem seems to be that the ApplicationDispatcherBootstrap class > produces an exception (that th

Re: Checkpoint metadata deleted by Flink after ZK connection issues

Hi Cristian, thanks for reporting this issue. It looks indeed like a very critical problem. The problem seems to be that the ApplicationDispatcherBootstrap class produces an exception (that the request job can no longer be found because of a lost ZooKeeper connection) which will be interpreted as

Re: Flink Batch Processing

Hi Sunitha, First and foremost, the DataSet API will be deprecated soon [1] so I would suggest trying to migrate to the DataStream API. When using the DataStream API it doesn't mean that you can not work with bounded inputs - you can. Flink SQL (Blink planner) is in fact using DataStream API to ex

Re: Flink SQL - can I force the planner to reuse a temporary view to be reused across multiple queries that access different fields?

Hi Dan, unfortunetely, it is very difficult to read you plan? Maybe you can share a higher resolution and highlight which part of the pipeline is A, B etc. In general, the planner should be smart enough to reuse subplans where appropriate. Maybe this is a bug or shortcoming in the optimizer r

Re: Scala: Static methods in interface require -target:jvm-1.8

Hi, It sounds more like an Intellij issue, not a Flink issue. But have you checked your configured target language level for your modules? Best regards, Piotrek pon., 28 wrz 2020 o 10:57 Lu Weizheng napisał(a): > Hi all, > > I recently upgraded Intellij IEDA from 2019 to 2020.2 Community Editi

Re: [DISCUSS] FLIP-144: Native Kubernetes HA for Flink

Thanks for creating this FLIP Yang Wang. I believe that many of our users will like a ZooKeeper-less HA setup. +1 for not separating the leader information and the leader election if possible. Maybe it is even possible that the contender writes his leader information directly when trying to obtain

Re: Dynamic Kafka Source

Hi Prasanna, As Theo has suggested on Stackoverflow, can you use multiple independent jobs instead? Piotrek sob., 26 wrz 2020 o 19:17 Prasanna kumar napisał(a): > Hi, > > My requirement has been captured by the following stack overflow question. > > > https://stackoverflow.com/questions/618768

Flink Batch Processing

Hi All, Need your help in Flink Batch processing: scenario described below: we have multiple vehicles, we get data from each vehicle at a very high speed, 1 record per minute.thresholds can be set by the owner for each vehicle.  Say: we have 3 vehicles, threshold is set for 2 vehicles. Vehicle 1,

Re: Flink Collector issue when Collection Object

Hi Anuj, I answered on SO. Let's keep the discussion there. On Mon, Sep 28, 2020 at 9:14 AM aj wrote: > > Hello All, > > Can somebody help me to resolve this and understand what is wrong i am > doing. > > > https://stackoverflow.com/questions/64063833/flink-collector-issue-when-collection-objec

Scala: Static methods in interface require -target:jvm-1.8

Hi all, I recently upgraded Intellij IEDA from 2019 to 2020.2 Community Edition. I didn’t do anything to Maven. My code could compile correctly before. But now I get the following error: Static methods in interface require -target:jvm-1.8 Probably because I use new WatermarkStrategy Scala API:

Flink Collector issue when Collection Object

Hello All, Can somebody help me to resolve this and understand what is wrong i am doing. https://stackoverflow.com/questions/64063833/flink-collector-issue-when-collection-object-with-map-of-object-class -- Thanks & Regards, Anuj Jain