Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

2023-01-30 Thread Yun Tang
Thanks Yuanfei for driving the frocksdb release! Best Yun Tang From: Yuan Mei Sent: Tuesday, January 31, 2023 15:09 To: Jing Ge Cc: Yanfei Lei ; d...@flink.apache.org ; user ; user...@flink.apache.org Subject: Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 relea

Re: "Error while retrieving the leader gateway" when using Kubernetes HA

2023-01-30 Thread Yang Wang
I assume you are using the standalone mode. Right? For the native K8s mode, the leader address should be *akka.tcp://flink@JM_POD_IP:6123/user/rpc/dispatcher_1 *when HA enabled. Best, Yang Anton Ippolitov via user 于2023年1月31日周二 00:21写道: > This is actually what I'm already doing, I'm only sett

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

2023-01-30 Thread Yuan Mei
Thanks Yanfei for driving the release! Best Yuan On Mon, Jan 30, 2023 at 8:46 PM Jing Ge via user wrote: > Hi Yanfei, > > Thanks for your effort. Looking forward to checking it. > > Best regards, > Jing > > On Mon, Jan 30, 2023 at 1:42 PM Yanfei Lei wrote: > >> It is very happy to announce the

Re: Using pyflink from flink distribution

2023-01-30 Thread Dian Fu
>> What is the reason for including opt/python/{pyflink.zip,cloudpickle.zip,py4j.zip} in the base distribution then? Oh, a guess: to make it easier for TaskManagers to run pyflink without having pyflink installed themselves? Somehow I'd guess this wouldn't work tho; I'd assume TaskManagers would

Processing watermarks in a broadcast connected stream

2023-01-30 Thread Sajjad Rizvi
Hi, I am trying to process watermarks in a BroadcastConnectedStream. However, I am not able to find any direct way to handle watermark events, similar to what we have in processWatermark1 in a KeyedCoProcessOperator. Following are further details. In the context of the example given in “A Pra

Re: "Error while retrieving the leader gateway" when using Kubernetes HA

2023-01-30 Thread Anton Ippolitov via user
This is actually what I'm already doing, I'm only setting high-availability: kubernetes myself. The other values are either defaults or set by the Operator: - jobmanager.rpc.port: 6123 is the default value (docs

Re: beam + flink + k8

2023-01-30 Thread Jan Lukavský
Hi, can you please share the command-line and complete output of the script? Are you using minikube? Can you share list of your running pods?  Jan On 1/30/23 14:25, P Singh wrote: Hi Team, I am trying to run beam job on top of flink on my local machine (kubernetes).  I have flink 1.14 an

Re: Using pyflink from flink distribution

2023-01-30 Thread Andrew Otto
Thanks Dian! > >> Is using pyflink from the flink distribution tarball (without pip) not a supported way to use pyflink? > You are right. What is the reason for including opt/python/{pyflink.zip,cloudpickle.zip,py4j.zip} in the base distribution then? Oh, a guess: to make it easier for TaskManag

Re: Which flink version is compatible with beam

2023-01-30 Thread Andrew Otto
Hi, I'm not sure about beam, but Flink is not officially compatible with python3.10. https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/installation/ > Python version (3.6, 3.7 or 3.8) is required for PyFlink. On Fri, Jan 27, 2023 at 11:50 PM P Singh wrote: > Hi, > > I

beam + flink + k8

2023-01-30 Thread P Singh
Hi Team, I am trying to run beam job on top of flink on my local machine (kubernetes). I have flink 1.14 and beam 2.43 images both running but when i submit the job it's not reaching to the flink cluster and getting failed with below error. ERROR:apache_beam.utils.subprocess_server:Starting job

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

2023-01-30 Thread Jing Ge via user
Hi Yanfei, Thanks for your effort. Looking forward to checking it. Best regards, Jing On Mon, Jan 30, 2023 at 1:42 PM Yanfei Lei wrote: > It is very happy to announce the release of FRocksDB 6.20.3-ververica-2.0. > > Compiled files for Linux x86, Linux arm, Linux ppc64le, MacOS x86, > MacOS ar

[ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

2023-01-30 Thread Yanfei Lei
It is very happy to announce the release of FRocksDB 6.20.3-ververica-2.0. Compiled files for Linux x86, Linux arm, Linux ppc64le, MacOS x86, MacOS arm, and Windows are included in FRocksDB 6.20.3-ververica-2.0 jar, and the FRocksDB in Flink 1.17 would be updated to 6.20.3-ververica-2.0. Release

Re: Which flink version is compatible with beam

2023-01-30 Thread bigdatadeveloper8
I have added flink:1.14 with Apache/beam:2.38.0When I submit the job python basic.py get warning:Root: make sure locally built python SDK docker image has python 3.8 interpreter. After this warning nothing is happening no logs or trace back. Please help. Sent from my Galaxy Original m