Re: Will broadcast stream affect performance because of the absence of operator chaining?

2019-08-06 Thread Wong Victor
Oops, accidentally sent the email. The good news is that you don’t have to checkpoint the state of the Kafka consumers. From: Wong Victor Date: Tuesday, August 6, 2019 at 11:31 PM To: Piotr Nowojski , 黄兆鹏 Cc: user Subject: Re: Will broadcast stream affect performance because of the absence of

Re: Will broadcast stream affect performance because of the absence of operator chaining?

2019-08-06 Thread Wong Victor
Hi, If the performance impact of braking the operator chain is huge, maybe you can read the latest schema from Kafka within the operators. It’s a little complicated, you have to start a Kafka consumer in e.g. ` RichFunction#open()` and reading from (the largest offset – 1), and handle new messa

Re: getting an exception

2019-08-05 Thread Wong Victor
Hi Avi: It seems you are submitting your job with an older Flink version (< 1.8), please check your flink-dist version. Regards, Victor From: Avi Levi Date: Monday, August 5, 2019 at 9:11 PM To: user Subject: getting an exception Hi, I'm using Flink 1.8.1. our code is mostly using Scala. Whe