Flink Collector issue when Collection Object

Hello All, Can somebody help me to resolve this and understand what is wrong i am doing. https://stackoverflow.com/questions/64063833/flink-collector-issue-when-collection-object-with-map-of-object-class -- Thanks & Regards, Anuj Jain

Scala: Static methods in interface require -target:jvm-1.8

Hi all, I recently upgraded Intellij IEDA from 2019 to 2020.2 Community Edition. I didn’t do anything to Maven. My code could compile correctly before. But now I get the following error: Static methods in interface require -target:jvm-1.8 Probably because I use new WatermarkStrategy Scala API:

Re: Flink Collector issue when Collection Object

Hi Anuj, I answered on SO. Let's keep the discussion there. On Mon, Sep 28, 2020 at 9:14 AM aj wrote: > > Hello All, > > Can somebody help me to resolve this and understand what is wrong i am > doing. > > > https://stackoverflow.com/questions/64063833/flink-collector-issue-when-collection-objec

Flink Batch Processing

Hi All, Need your help in Flink Batch processing: scenario described below: we have multiple vehicles, we get data from each vehicle at a very high speed, 1 record per minute.thresholds can be set by the owner for each vehicle.  Say: we have 3 vehicles, threshold is set for 2 vehicles. Vehicle 1,

Re: Dynamic Kafka Source

Hi Prasanna, As Theo has suggested on Stackoverflow, can you use multiple independent jobs instead? Piotrek sob., 26 wrz 2020 o 19:17 Prasanna kumar napisał(a): > Hi, > > My requirement has been captured by the following stack overflow question. > > > https://stackoverflow.com/questions/618768

Re: [DISCUSS] FLIP-144: Native Kubernetes HA for Flink

Thanks for creating this FLIP Yang Wang. I believe that many of our users will like a ZooKeeper-less HA setup. +1 for not separating the leader information and the leader election if possible. Maybe it is even possible that the contender writes his leader information directly when trying to obtain

Re: Scala: Static methods in interface require -target:jvm-1.8

Hi, It sounds more like an Intellij issue, not a Flink issue. But have you checked your configured target language level for your modules? Best regards, Piotrek pon., 28 wrz 2020 o 10:57 Lu Weizheng napisał(a): > Hi all, > > I recently upgraded Intellij IEDA from 2019 to 2020.2 Community Editi

Re: Flink SQL - can I force the planner to reuse a temporary view to be reused across multiple queries that access different fields?

Hi Dan, unfortunetely, it is very difficult to read you plan? Maybe you can share a higher resolution and highlight which part of the pipeline is A, B etc. In general, the planner should be smart enough to reuse subplans where appropriate. Maybe this is a bug or shortcoming in the optimizer r

Re: Flink Batch Processing

Hi Sunitha, First and foremost, the DataSet API will be deprecated soon [1] so I would suggest trying to migrate to the DataStream API. When using the DataStream API it doesn't mean that you can not work with bounded inputs - you can. Flink SQL (Blink planner) is in fact using DataStream API to ex

Re: Checkpoint metadata deleted by Flink after ZK connection issues

Hi Cristian, thanks for reporting this issue. It looks indeed like a very critical problem. The problem seems to be that the ApplicationDispatcherBootstrap class produces an exception (that the request job can no longer be found because of a lost ZooKeeper connection) which will be interpreted as

Re: Checkpoint metadata deleted by Flink after ZK connection issues

Hi all, I will have a look. Kostas On Mon, Sep 28, 2020 at 3:56 PM Till Rohrmann wrote: > > Hi Cristian, > > thanks for reporting this issue. It looks indeed like a very critical problem. > > The problem seems to be that the ApplicationDispatcherBootstrap class > produces an exception (that th

Streaming SQL Job Switches to FINISHED before all records processed

Hey all, I'm not sure if I've missed something in the docs, but I'm having a bit of trouble with a streaming SQL job that starts w/ raw SQL queries and then transitions to a more traditional streaming job. I'm on Flink 1.10 using the Blink planner, running locally with no checkpointing. The job l

Re: [DISCUSS] FLIP-144: Native Kubernetes HA for Flink

Hi Till, thanks for your valuable feedback. 1. Yes, leader election and storing leader information will use a same ConfigMap. When a contender successfully performs a versioned annotation update operation to the ConfigMap, it means that it has been elected as the leader. And it will write the lead