[DISCUSS] Contribution of Multi Cluster Kafka Source

Hi community, We have been working on a Multi Cluster Kafka Source and are looking to contribute it upstream. I've given a talk about the features and design at a Flink meetup: https://youtu.be/H1SYOuLcUTI. The main features that it provides is: 1. Reading multiple Kafka clusters within a single

Flink 1.12 StreamRecordQueueEntry is not public class

Hi I am trying to upgrade the version to 1.12. Some of the old code is using org.apache.flink.streaming.api.operators.async.queue.StreamRecordQueueEntry; This is no longer public in 1.12. Any pointers as to how to work around this? Thanks, Milind

Re: Is it possible to use DataStream API keyBy followed by Table API SQL in PyFlink?

Sorry for the noise, I completely missed this part of the documentation describing exactly how to do this: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/data_stream_api/ From: John Tipper Sent: 23 June 2022 21:35 To: user@flink.apache

Is it possible to use DataStream API keyBy followed by Table API SQL in PyFlink?

Hi all, In PyFlink, is it possible to use the DataStream API to create a DataStream by means of StreamExecutionEnvironment's addSource(...), then perform transformations on this data stream using the DataStream API and then convert that stream into a form where SQL statements can be executed on

Re: slack invite link

Hi Sweta I see that you've ready manage to join, but for everyone else, you can get access as documented at https://flink.apache.org/gettinghelp.html#slack Best regards, Martijn Op do 23 jun. 2022 om 20:20 schreef Sweta Kalakuntla < skalakun...@bandwidth.com> > Hi, > > Please add me to the sla

Re: slack invite link

Hi, Please add me to the slack channel. Thanks, Sweta On Sun, Jun 5, 2022 at 6:42 AM Jing Ge wrote: > Hi, > > Invites have been sent. Please check your emails. Thanks! > > Regards, > JIng > > On Sun, Jun 5, 2022 at 12:38 PM Jay Ghiya wrote: > >> Request community to share invite link for me a

Re: HTTP 404 while creating resource with flink kubernetes operator and frabric8 client

Yes I did install the operator as per the instructions. Verified the installation. I removed the IP address from the URL for security reasons. DO we need to install any other additional components? kubectl get pods NAMEREADY STATUSRESTARTS AGE flin

Re: Advice needed: Flink Kubernetes Operator with Prometheus Configuration

Hi Morgan, There is a placeholder in the values.yaml: [image: image.png] This should create an entry on the operator container: [image: image.png] Can you share the output of this command pls? k get deploy flink-kubernetes-operator -o yaml Thanks, Matyas On Thu, Jun 23, 2022 at 12:12 PM Ge

Advice needed: Flink Kubernetes Operator with Prometheus Configuration

Greetings all, I am trying to deploy Flink jobs using the Flink Kubernetes Operator and I would like to have Prometheus scrape metrics from the various pods. The jobs are created successfully, however, the metrics don't appear to be available. The following steps were followed based on the

Re: Flink operator - ignore ssl cert validation

I guess the problem is caused by the self-signed certificates, and Calvin is looking for a way to disable the SSL certificate validation: something like this http://www.java2s.com/Code/Java/Network-Protocol/DisablingCertificateValidationinanHTTPSConnection.htm On Thu, Jun 23, 2022 at 11:40 AM Yang

Re: Flink operator - ignore ssl cert validation

Do you mean the HttpArtifactFetcher could not support HTTPS? cc @Aitozi Best, Yang calvin beloy 于2022年6月22日周三 22:10写道： > Sorry typo "jarring" should be "jar url". > > Sent from Yahoo Mail on Android >

Re: Flink Operator - Support for k8s HA jobmanager

Matyas's answer is on the point. You need to mount a shared volume for all the JobManager pods so that the uploaded jars are visible for them all. Best, Yang Őrhidi Mátyás 于2022年6月23日周四 04:34写道： > I guess the problem here is that your JM pods do not have access to a > common upload folder. You

Re: HTTP 404 while creating resource with flink kubernetes operator and frabric8 client

Do you have installed the operator along with CRD[1]? [1]. https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.0/docs/try-flink-kubernetes-operator/quick-start/#deploying-the-operator Best, Yang yu'an huang 于2022年6月23日周四 13:04写道： > Hi， > > It seems that you can't find t

Re: Flink k8s Operator on AWS?

Thanks for your valuable inputs. To make deploying Flink on K8s easy as a normal Java application is certainly the mission of Flink Kubernetes Operator. Obviously, we are still a little far from this mission. Back to the user jars download, I think it makes sense to introduce the artifact fetcher