Re: Apache Flink - User Defined Functions - Exception when passing all arguments

2022-02-22 Thread M Singh
Thanks FG for your recommendations.  Let me try that.  Thanks for your time. On Thursday, February 17, 2022, 04:40:07 AM EST, Francesco Guardiani wrote: Hi, The SQL syntax is not supported, as the SQL standard itself does not allow it. It sounds strange that it fails at validation phas

Apache Flink - Exception on left outer join with 'kafka' connector

2022-02-22 Thread M Singh
Hi Folks: I am using 'kafka' connector and joining with data from jdbc source (using connector).   I am using Flink v 1.14.3.  If I do a left outer join between kafka source and jdbc source, and try to save it to another kafka sink using connectors api, I get the following exception: Exception i

Re: Pyflink with pulsar

2022-02-22 Thread Dian Fu
Hi Ananth, It's already code freeze for 1.15 and you can refer to [1] for more details. Regards, Dian [1] https://www.mail-archive.com/dev@flink.apache.org/msg54262.html On Sun, Feb 20, 2022 at 1:51 AM Ananth Gundabattula < agundabatt...@darwinium.com> wrote: > Thanks a lot Wong. > > > > I was

Re: Cannot upgrade helm chart

2022-02-22 Thread Frank Dekervel
Hello, And if you want to go for a deployment, consider using a statefulset instead. This way you are certain that, on upgrade, it will let the old version exit before it starts the new version. Greetings, Frank On 22.02.22 00:23, Austin Cawley-Edwards wrote: Hey Marco, There’s unfortuna

Re: No effect from --allowNonRestoredState or "execution.savepoint.ignore-unclaimed-state" in K8S application mode

2022-02-22 Thread Andrey Bulgakov
Thank you, Yang. That was it! Specifying "--fromSavepoint" and "--allowNonRestoredState" for "run-application" together did the trick. I was a bit confused, because when you run "flink run-application --help", it only tells you about the "--executor" and "--target" options. So I assumed I should p

java.lang.Exception: Job leader for job id 0efd8681eda64b072b72baef58722bc0 lost leadership.

2022-02-22 Thread Jai Patel
We are developing a new feature for our Flink application that relies upon joining multiple Kafka Streams and uses Flink State to handle joining information asynchronously. Recently as the volume of data has been growing, we've been noticing a couple exceptions while trying to enable the feature.

Re: DataStream API: Parquet File Format with Scala Case Classes

2022-02-22 Thread Ryan van Huuksloot
Hi Fabian, Thanks for the response! I'll take a look at the CSVReaderFormat. Our team is interested in contributing to Parquet. However, our capacity for the current sprint is fully committed to other workstreams. I'll put this issue onto the backlog and see how it stacks against our internal pri

Avro Flink Row Conversion

2022-02-22 Thread Haizhou Zhao
Hello Flink Community, I’m looking for an interface to convert Avro records to Flink row. To be more specific, I’m aware there is this interface that converts serialized Avro records to Flink rows ( https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flin

Trouble sinking to Kafka

2022-02-22 Thread Marco Villalobos
I keep on receiving this exception during the execution of a simple job that receives time series data via Kafka, transforms it into avro format, and then sends into a Kafka topic consumed by druid. Any advise would be appreciated as to how to resolve this type of error. I'm using Apache Kafka

Re: Flink 1.15 deduplication view and lookup join

2022-02-22 Thread Francis Conroy
Hi Yun, The joined data is the versioned table in this case, I managed to get it as far as fixing all of the static errors but the temporal join just doesn't have a result... No idea what's going on. In reality I don't think we even want a temporal join, we just want to add a few extra columns to

Joining a flink materialised view table with a streaming table

2022-02-22 Thread Francis Conroy
Hi all, I recently put up a question about a deduplication query related to a join and realised that I was probably asking the wrong question. I'm using Flink 1.15-SNAPSHOT (97ddc39945cda9bf1f52ab159852fdb606201cf2) as we're using the RabbitMQ connector with pyflink. We won't go to prod until 1.15

Re: No effect from --allowNonRestoredState or "execution.savepoint.ignore-unclaimed-state" in K8S application mode

2022-02-22 Thread Yang Wang
The config options configured by -D param should take effect. It is also the recommended way instead of CLI options(e.g. --fromSavepoint). Not only the K8s application, it also does not work for yarn application and yarn per-job mode. I believe it is indeed a bug in the current implementation and h

Flink metrics via permethous or opentelemerty

2022-02-22 Thread Sigalit Eliazov
Hello. I am looking for a way to expose flink metrics via opentelemerty to the gcp could monitoring dashboard. Does anyone has experience with that? If it is not directly possible we thought about using permethous as a middlewere. If you have experience with that i would appreciate any guidance.