Re: Cannot run pyflink example using Flink CLI

Great, thanks! Kind regards, Levan Huyen On Fri, 21 Oct 2022 at 00:53, Biao Geng wrote: > You are right. > It contains the python package `pyflink` and some dependencies like py4j > and cloudpickle but does not contain all relevant dependencies(e.g. > `google.protobuf` as the error log shows, w

Re: Cannot run pyflink example using Flink CLI

You are right. It contains the python package `pyflink` and some dependencies like py4j and cloudpickle but does not contain all relevant dependencies(e.g. `google.protobuf` as the error log shows, which I also reproduce in my own machine). Best, Biao Geng Levan Huyen 于2022年10月20日周四 19:53写道： >

Re: Cannot run pyflink example using Flink CLI

Thanks Biao. May I ask one more question: does the binary package on Apache site (e.g: https://archive.apache.org/dist/flink/flink-1.15.2) contain the python package `pyflink` and its dependencies? I guess the answer is no. Thanks and regards, Levan Huyen On Thu, 20 Oct 2022 at 18:13, Biao Geng

Re: Cannot run pyflink example using Flink CLI

Hi Levan, Great to hear that your issue is resolved! For the follow-up question, I am not quite familiar with AWS EMR's configuration for flink but due to the error you attached, it looks like that pyflink may not ship some 'Google' dependencies in the Flink binary zip file and as a result, it will

Re: Cannot run pyflink example using Flink CLI

Hi Biao, Thanks for your help. That solved my issue. It turned out that in setup1 (in EMR), I got apache-flink installed, but the package (and its dependencies) are not in the directory `/usr/lib/python3.7/site-packages` (corresponding to the python binary in `/usr/bin/python3`). For some reason,

Re: Cannot run pyflink example using Flink CLI

Hi Levan, For your setup1 & 2, it looks like the python environment is not ready. Have you tried python -m pip install apache-flink for the first 2 setups? For your setup3, as you are trying to use `flink run ...` command, it will try to connect to a launched flink cluster but I guess you did not

Cannot run pyflink example using Flink CLI

Hi, I'm new to PyFlink, and I couldn't run a basic example that shipped with Flink. This is the command I tried: ./bin/flink run -py examples/python/datastream/word_count.py Here below are the results I got with different setups: 1. On AWS EMR 6.8.0 (Flink 1.15.1): *Error: No module named 'goog

Re: Flink CLI - pass command line arguments to a pyflink job

gt;> >>> Matthias >>> >>> On Mon, Nov 22, 2021 at 3:58 PM Kamil ty wrote: >>> >>>> Hey, >>>> >>>> Looking at the examples at Command-Line Interface | Apache Flink >>>> <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/> >>>> I >>>> don't see an example of passing command line arguments to a pyflink job >>>> when deploying the job to a remote cluster with flink cli. Is this >>>> supported? >>>> >>>> Best Regards >>>> Kamil >>>> >>>

Re: Flink CLI - pass command line arguments to a pyflink job

nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/> >>> I >>> don't see an example of passing command line arguments to a pyflink job >>> when deploying the job to a remote cluster with flink cli. Is this >>> supported? >>> >>> Best Regards >>> Kamil >>> >>

Re: Flink CLI - pass command line arguments to a pyflink job

:58 PM Kamil ty wrote: > >> Hey, >> >> Looking at the examples at Command-Line Interface | Apache Flink >> <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/> >> I >> don't see an example of passing command line argum

Re: Flink CLI - pass command line arguments to a pyflink job

when deploying the job to a remote cluster with flink cli. Is this > supported? > > Best Regards > Kamil >

Flink CLI - pass command line arguments to a pyflink job

Hey, Looking at the examples at Command-Line Interface | Apache Flink <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/> I don't see an example of passing command line arguments to a pyflink job when deploying the job to a remote cluster with flink c

Re: Using the flink CLI option --pyRequirements

27;ve > built 1.15-SNAPSHOT@7578758fa8c84314b8b3206629b3afa9ff41b636 and have run > the wordcount example, everything else seems to work, I just can't submit a > pyflink job to my cluster when using the --pyRequirements option. > > I started going down the line of debugging the flink CLI using int

Using the flink CLI option --pyRequirements

just can't submit a pyflink job to my cluster when using the --pyRequirements option. I started going down the line of debugging the flink CLI using intellij idea, but wasn't able to figure out how to make my venv with pyflink installed available to the debug environment. Thanks, Francis Co

Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment

Hi Yang, Thanks for the reply, those information is very helpful. Best, Fuyao From: Yang Wang Date: Tuesday, April 6, 2021 at 01:11 To: Fuyao Li Cc: user Subject: Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment Hi Fuyao, Sorry for the late reply

Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment

help of ZooKeeper. In terms > of this, native k8s is better. Any other ideas? Thanks for your help. > > > > Best, > > Fuyao > > > > *From: *Fuyao Li > *Date: *Thursday, April 1, 2021 at 12:22 > *To: *Yang Wang > *Cc: *user > *Subject: *Re: [External] : R

Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment

for your help. Best, Fuyao From: Fuyao Li Date: Thursday, April 1, 2021 at 12:22 To: Yang Wang Cc: user Subject: Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment Hi Yang, Thanks for sharing the insights. For problem 1: I think I can’t do telnet in the

Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment

://github.com/GoogleCloudPlatform/flink-on-k8s-operator [2] https://github.com/lyft/flinkk8soperator [3] https://youtu.be/pdFPr_VOWTU Best, Fuyao From: Yang Wang Date: Tuesday, March 30, 2021 at 19:15 To: Fuyao Li Cc: user Subject: Re: [External] : Re: Need help with executing Flink CLI for native

Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment

network configuration, but failed to find a reason, could you give me some >hints? >3. In production, what is the suggested approach to list and cancel >jobs? The current manual work of “kubectl exec” into pods is not very >reliable.. How to automate this process

Re: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment

thanks. Best, Fuyao From: Yang Wang Date: Monday, March 29, 2021 at 20:40 To: Fuyao Li Cc: user Subject: [External] : Re: Need help with executing Flink CLI for native Kubernetes deployment Hi Fuyao, Thanks for trying the native Kubernetes integration. Just like you know, the Flink rest serv

Re: Need help with executing Flink CLI for native Kubernetes deployment

Hi Fuyao, Thanks for trying the native Kubernetes integration. Just like you know, the Flink rest service could be exposed in following three types, configured via "kubernetes.rest-service.exposed.type". * ClusterIP, which means you could only access the Flink rest endpoint inside the K8s cluste

Need help with executing Flink CLI for native Kubernetes deployment

Hi Community, Yang, I am new to Flink on native Kubernetes and I am trying to do a POC for native Kubernetes application mode on Oracle Cloud Infrastructure. I was following the documentation here step by step: [1] I am using Flink 1.12.1, Scala 2.11, java 11. I was able to create a native Kube

Re: Flink cli to upload flink jar and run flink jobs in the Jenkins pipeline

ress from ZK. > > Please ensure that you have specified the HA related config options in CLI > via -D or set them in the flink-conf.yaml. > > Best, > Yang > > sidhant gupta 于2021年2月3日周三 下午10:02写道： > >> Is it possible to use flink CLI instead of flink client for connectin

Re: Flink cli to upload flink jar and run flink jobs in the Jenkins pipeline

them in the flink-conf.yaml. Best, Yang sidhant gupta 于2021年2月3日周三 下午10:02写道： > Is it possible to use flink CLI instead of flink client for connecting > zookeeper using network load balancer to retrieve the leader Jobmanager > address? > > On Wed, Feb 3, 2021, 12:42 PM Yang Wang

Re: Job submission failure via flink cli

Please make sure the client and server version are in sync. On 2/3/2021 4:12 PM, sidhant gupta wrote: I am getting following error while running the below command with the attached conf/flink-conf.yaml: bin/flink run -c firstflinkpackage.someJob ../somejob.jar arg1 arg2 arg3 2021-02-03 15:04

Job submission failure via flink cli

I am getting following error while running the below command with the attached conf/flink-conf.yaml: bin/flink run -c firstflinkpackage.someJob ../somejob.jar arg1 arg2 arg3 2021-02-03 15:04:24,113 INFO org.apache.flink.runtime.dispatcher. StandaloneDispatcher [] - Received JobGraph submission 9

Re: Flink cli to upload flink jar and run flink jobs in the Jenkins pipeline

Is it possible to use flink CLI instead of flink client for connecting zookeeper using network load balancer to retrieve the leader Jobmanager address? On Wed, Feb 3, 2021, 12:42 PM Yang Wang wrote: > I think the Flink client could make a connection with ZooKeeper via the > networ

Re: Flink cli to upload flink jar and run flink jobs in the Jenkins pipeline

; > , > we can provide zookeeper address in the flink cli arguments and it would > upload/ submit the jar to the leader jobmanager. But since I am using > network load balancer in front of zookeeper, I guess it is not able to make > connection with the zookeeper. Please provide s

Flink cli to upload flink jar and run flink jobs in the Jenkins pipeline

[1] <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Best-way-to-find-the-current-alive-jobmanager-with-HA-mode-zookeeper-td21787.html> , we can provide zookeeper address in the flink cli arguments and it would upload/ submit the jar to the leader jobmanager. But since I am

RE: Flink cli Stop command exception

(Nokia - IN/Bangalore) ; user@flink.apache.org Subject: Re: Flink cli Stop command exception Maybe FLINK-16626[1] is related. And it is fixed in 1.10.1 and 1.11. [1]. https://issues.apache.org/jira/browse/FLINK-16626 Best, Yang Yun Tang mailto:myas...@live.com>> 于2020年12月10日周四 上午11:06

Re: Flink cli Stop command exception

he savepoint failed to complete. > > Best > Yun Tang > -- > *From:* V N, Suchithra (Nokia - IN/Bangalore) > *Sent:* Wednesday, December 9, 2020 23:45 > *To:* user@flink.apache.org > *Subject:* Flink cli Stop command exception > > > He

Re: Flink cli Stop command exception

Subject: Flink cli Stop command exception Hello, I am running streaming flink job and I was using cancel command with savepoint to cancel the job. From flink 1.10 version stop command should be used instead of cancel command. But I am getting below error sometimes. Please let me know what

Flink cli Stop command exception

Hello, I am running streaming flink job and I was using cancel command with savepoint to cancel the job. From flink 1.10 version stop command should be used instead of cancel command. But I am getting below error sometimes. Please let me know what might be the issue. {"host":"cancel1-flinkcli

Re: Missing help about run-application action in Flink CLI client

Here it is: https://issues.apache.org/jira/browse/FLINK-19969 Best, Flavio On Wed, Nov 4, 2020 at 11:08 AM Kostas Kloudas wrote: > Could you also post the ticket here @Flavio Pompermaier and we will > have a look before the upcoming release. > > Thanks, > Kostas > > On Wed, Nov 4, 2020 at 10:5

Re: Missing help about run-application action in Flink CLI client

Could you also post the ticket here @Flavio Pompermaier and we will have a look before the upcoming release. Thanks, Kostas On Wed, Nov 4, 2020 at 10:58 AM Chesnay Schepler wrote: > > Good find, this is an oversight in the CliFrontendParser; no help is > printed for the run-application action.

Re: Missing help about run-application action in Flink CLI client

Good find, this is an oversight in the CliFrontendParser; no help is printed for the run-application action. Can you file a ticket? On 11/4/2020 10:53 AM, Flavio Pompermaier wrote: Hello everybody, I was looking into currently supported application-modes when submitting a Flink job so I tried

Missing help about run-application action in Flink CLI client

Hello everybody, I was looking into currently supported application-modes when submitting a Flink job so I tried to use the CLI help (I'm using Flink 11.0) but I can't find any help about he action "run-application" at the moment...am I wrong? Is there any JIRA to address this missing documentation

Re: (DISSCUSS) flink cli need load '--classpath' files

t;>> > at com.netease.java.TopSpeedWindowing.main(TopSpeedWindowing.java:98) >>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> > at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> > at >&

Re: (DISSCUSS) flink cli need load '--classpath' files

n.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> > at java.lang.reflect.Method.invoke(Method.java:498) >>> > at >>> > >>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) >>> > at >>> > >>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) >>> > at >>> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) >>> > at >>> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)” >>> >As I read the code , flink cli have not load the —classspath jar, >>> So It seems >>> > a bug about the flink cli. Are you agree with me? >>> > Best, >>> > Ouywl >>> > >>> >> > > -- > Best, Jingsong Lee >

Re: (DISSCUSS) flink cli need load '--classpath' files

I think tison's answer is on point. All the Flink cli options should be specified before the user jar. We have a very clear help message. Syntax: run [OPTIONS] Best, Yang Jingsong Li 于2020年3月6日周五 下午10:27写道： > Hi tison and Aljoscha, > > Do you think "--classpath can not

Re: (DISSCUSS) flink cli need load '--classpath' files

gt;> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) >> > at >> > >> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) >> > at >> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) >> > at >> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)” >> >As I read the code , flink cli have not load the —classspath jar, So >> It seems >> > a bug about the flink cli. Are you agree with me? >> > Best, >> > Ouywl >> > >> > -- Best, Jingsong Lee

Re: (DISSCUSS) flink cli need load '--classpath' files

gt; > at >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > at >> > >> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) >> > at >> > >> org.apa

Re: (DISSCUSS) flink cli need load '--classpath' files

nk.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) > > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) > > at > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)” > >As I read the code , flink cli have not load the —classspath jar, So > It seems > > a bug about the flink cli. Are you agree with me? > > Best, > > Ouywl > > >

Re: (DISSCUSS) flink cli need load '--classpath' files

t; > at > org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274) > > at > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)” > >As I read the code , flink cli have not load the —classspath jar, So > It seems > > a bug about the flink cli. Are you agree with me? > > Best, > > Ouywl > > > -- Best, Jingsong Lee

Re: (DISSCUSS) flink cli need load '--classpath' files

org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)” As I read the code , flink cli have not load the —classspath jar, So It seems a bug about the flink cli. Are you agree with me? Best, Ouywl

(DISSCUSS) flink cli need load '--classpath' files

org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)” As I read the code , flink cli have not load the —classspath

Re: new user does not run job use flink cli

Hi, Do you mean job submission is OK with local user name "flink", but not for other users? Have you ever checked the authorization of "hdfs://user/flink/recovery"? I guess other users do not have the access right. &#38;#38;#38;#38;#10084; <799326...@qq.com> 于2019年7月11日周四 上午11:55写道： > flink

new user does not run job use flink cli

flink-conf.yaml jobmanager.heap.size: 1024m taskmanager.heap.size: 6144m taskmanager.numberOfTaskSlots: 3 parallelism.default: 1 high-availability: zookeeper high-availability.zookeeper.quorum: 10.1.1.15:2181,10.1.1.16:2181,10.1.1.17:2181 high-availability.zookeeper.path.root: /flink high-availabil

Flink CLI distributed cache fault

function and sink 4. Execute job from Flink CLI (Windows/Linux) In order to restore job from savepoint or from checkpoint we need to run our job from Flink CLI. And pipelines that have distributed cache fails their execution. Moreover it is different in Linux and Windows systems: in Windows we

Re: Flink CLI

: > Hello! > > I am working on automating our deployments to our Flink cluster. I had a > couple questions about the flink cli. > > 1) I thought there was an "update" command that would internally manage > the cancel with savepoint, upload new jar, restart from savepo

Re: Flink CLI

t; Oytun Tez > > *M O T A W O R D* > The World's Fastest Human Translation Platform. > oy...@motaword.com — www.motaword.com > > > On Wed, Apr 24, 2019 at 11:06 AM Steven Nelson > wrote: > >> Hello! >> >> I am working on automating our deployments to

Re: Flink CLI

> wrote: > > > Hello! > > > > > > I am working on automating our deployments to our Flink cluster. I had a > > > couple questions about the flink cli. > > > > > > 1) I thought there was an "update" command that would internally manage &g

Re: Flink CLI

e questions about the flink cli. > > 1) I thought there was an "update" command that would internally manage > the cancel with savepoint, upload new jar, restart from savepoint process. > > 2) Is there a way to get the Flink cli to output it's result in a json > form

Re: Flink CLI

Hi Steve, I recently solved this problem using the REST api and some python scripts. The script has a function "upgrade_job" which will cancel with savepoint, optionally upload a new jar from the local filestystem or S3, and start the job from the savepoint including any changes in parallelism.

Flink CLI

Hello! I am working on automating our deployments to our Flink cluster. I had a couple questions about the flink cli. 1) I thought there was an "update" command that would internally manage the cancel with savepoint, upload new jar, restart from savepoint process. 2) Is there a way

Re: Breakage in Flink CLI in 1.5.0

Hi Till: So how can we get the right rest address and port when using HA mode on Yarn? I get it from the rest api "/jars ". But when I submit a job use the flink run -m ,it failed . org.apache.flink.client.program.ProgramInvocationException: Could not retrieve the

Re: Flink CLI does not return after submitting yarn job in detached mode

t; job in detached mode. I tried that this morning but it also didn't work. > > flink CLI always starts in interactive mode, and somehow ignores > option -yd specified in yarn-cluster mode. Can someone verify this please? > If its the case, its a bug in flink CLI. > >

Re: Flink CLI does not return after submitting yarn job in detached mode

Hi Vino, We should not have to specify class name using -c option to run job in detached mode. I tried that this morning but it also didn't work. flink CLI always starts in interactive mode, and somehow ignores option -yd specified in yarn-cluster mode. Can someone v

Re: Flink CLI does not return after submitting yarn job in detached mode

n detached mode. as >> per the docs for flink 1.4.2, I am using -yd to do that. >> >> The problem I am having is the flink bash script doesn't terminate >> execution and return until I press control + c. In detached mode, I would >> expect the flink CLI to return

Re: Flink CLI does not return after submitting yarn job in detached mode

6日周四 上午5:01写道： > Hi there, > > I am trying to run a single flink job on YARN in detached mode. as per > the docs for flink 1.4.2, I am using -yd to do that. > > The problem I am having is the flink bash script doesn't terminate > execution and return until I press control + c.

Flink CLI does not return after submitting yarn job in detached mode

Hi there, I am trying to run a single flink job on YARN in detached mode. as per the docs for flink 1.4.2, I am using -yd to do that. The problem I am having is the flink bash script doesn't terminate execution and return until I press control + c. In detached mode, I would expect the

Re: How to submit a job with dependency jars by flink cli in Flink 1.4.2?

Hi, I’m glad that you have found a solution to your problem :) To shorten feedback you can/should test as much logic as possible using smaller unit tests and some small scale integration tests: https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/testing.html

Re: How to submit a job with dependency jars by flink cli in Flink 1.4.2?

Hi, -yt,--yarnship Ship files in the specified directory (t for transfer) I guess that you even got a warning in your log files: LOG.warn("Ship directory is not a directory. Ignoring it.”); I’m not sure, but maybe with `-yt` you do not

Re: How to submit a job with dependency jars by flink cli in Flink 1.4.2?

> 在 2018年8月3日，19:03，Joshua Fan 写道： > > Hi, > > I'd like to submit a job with dependency jars by flink run, but it failed. > > Here is the script, > > /usr/bin/hadoop/software/flink-1.4.2/bin/flink run \ > -m yarn-cluster -yn 1 -ys 8 -yjm 2148 -ytm 4096 -ynm jarsTest \ > -c StreamExample \ >

Re: How to submit a job with dependency jars by flink cli in Flink 1.4.2?

Hi, Are those paths: file: '/home/work/xxx/lib/commons-math3-3.5.jar' (missing) file: '/home/work/xxx/lib/flink-connector-kafka-0.8_2.11-1.4.2.jar' (missing) accessible from the inside of your container? bin/flink run --help (…) -C,--classpath Adds a URL to each u

How to submit a job with dependency jars by flink cli in Flink 1.4.2?

Hi, I'd like to submit a job with dependency jars by flink run, but it failed. Here is the script, /usr/bin/hadoop/software/flink-1.4.2/bin/flink run \ -m yarn-cluster -yn 1 -ys 8 -yjm 2148 -ytm 4096 -ynm jarsTest \ -c StreamExample \ -C file:/home/work/xxx/lib/commons-math3-3.5.jar \ -C file:/h

Re: Flink CLI properties with HA

Vino, I'm not getting any error but my suspicion was that if I dont specify this `high-availability.storageDir` property in flink CLI side then the CLI will not be able to submit job to flink cluster(HA enabled). But if provide this property in CLI side the job submission will be successful

Re: Flink CLI properties with HA

ath, > > It seems Flink CLI for standalone would not access > *high-availability.storageDir.* > > What's the exception stack trace in your environment? > > Thanks, vino. > > 2018-07-17 15:08 GMT+08:00 Sampath Bhat : > >> Hi vino >> >> Should the

Re: Flink CLI properties with HA

Hi Sampath, It seems Flink CLI for standalone would not access *high-availability.storageDir.* What's the exception stack trace in your environment? Thanks, vino. 2018-07-17 15:08 GMT+08:00 Sampath Bhat : > Hi vino > > Should the flink CLI have access to the path mentio

Re: Flink CLI properties with HA

Hi vino Should the flink CLI have access to the path mentioned in *high-availability.storageDir*? If my flink cluster is on set of machines and i submit my job from flink CLI from another independent machine by giving necessary details will the CLI try to access *high-availability.storageDir

Re: Flink CLI properties with HA

Hi Sampath, Flink CLI need to retrieve the JobManager leader address, so it need to access the HA specific configuration. Because if based on Zookeeper to implement the HA, the leader address information will fetch from Zookeeper. The main use of config item *high-availability.storageDir* is

Fwd: Flink CLI properties with HA

-- Forwarded message -- From: Sampath Bhat Date: Fri, Jul 13, 2018 at 3:18 PM Subject: Flink CLI properties with HA To: user Hello When HA is enabled in the flink cluster and if I've to submit job via flink CLI then in the flink-conf.yaml of flink CLI should contain

Flink CLI properties with HA

Hello When HA is enabled in the flink cluster and if I've to submit job via flink CLI then in the flink-conf.yaml of flink CLI should contain this properties - high-availability: zookeeper high-availability.cluster-id: flink high-availability.zookeeper.path.root: flink

Re: Breakage in Flink CLI in 1.5.0

rvices( >>> >>> HighAvailabilityServicesUtils.java:83) >>> >>> > at >>> >>> > org.apache.flink.client.program.ClusterClient.( >>> >>> ClusterClient.java:158) >>> >>> > at >>

Re: Breakage in Flink CLI in 1.5.0

apache.flink.client.program.rest.RestClusterClient.( >> >>> RestClusterClient.java:184) >> >>> > at >> >>> > org.apache.flink.client.program.rest.RestClusterClient.( >> >>> RestClusterClient.java:157) >> >>> &g

Re: Breakage in Flink CLI in 1.5.0

... 7 more > >>> > > >>> > > >>> > On Wed, Jun 20, 2018 at 11:18 AM, Sampath Bhat < > >>> sam414255p...@gmail.com> > >>> > wrote: > >>> > > >>> >> Hi Chesnay > &

Re: Breakage in Flink CLI in 1.5.0

> Hi Chesnay >>> >> >>> >> If REST API (i.e. the web server) is mandatory for submitting jobs >>> then >>> >> why is there an option to set rest.port to -1? I think it should be >>> >> mandatory to set some valid port for rest.po

Re: Breakage in Flink CLI in 1.5.0

e must be some way to submit jobs even if REST API (i.e. the web >> >> server) is not instantiated. >> >> >> >> If jobmanger.rpc.address is not required for flink client then why is >> it >> >> still looking for that property in flink-conf.yaml?

Re: Breakage in Flink CLI in 1.5.0

address is still *required *due to some technical implementations; it >>> may be that you can set this to some arbitrary value however. >>> >>> As a result the REST API (i.e. the web server) must be running in order >>> to submit jobs

Re: Breakage in Flink CLI in 1.5.0

e to submit the job. > >> > >> > >> On Tue, Jun 19, 2018 at 5:49 PM, Chesnay Schepler > >> wrote: > >> > >>> In 1.5 we reworked the job-submission to go through the REST API > instead > >>> of akka. > >>> > >

Re: Breakage in Flink CLI in 1.5.0

ll *required *due to some technical implementations; it may be that you can set this to some arbitrary value however. As a result the REST API (i.e. the web server) must be running in order to submit jobs. On 19.06.2018 14:12, Sampath Bhat wrote: Hello I'm using Flink 1.5.0 version and Flink CLI t

Re: Breakage in Flink CLI in 1.5.0

rary value however. >> >> As a result the REST API (i.e. the web server) must be running in order >> to submit jobs. >> >> >> On 19.06.2018 14:12, Sampath Bhat wrote: >> >> Hello >> >> I'm using Flink 1.5.0 version and Flink CLI to submit

Re: Breakage in Flink CLI in 1.5.0

2, Sampath Bhat wrote: > > Hello > > I'm using Flink 1.5.0 version and Flink CLI to submit jar to flink cluster. > > In flink 1.4.2 only job manager rpc address and job manager rpc port were > sufficient for flink client to connect to job manager and submit the job. > &

Re: Breakage in Flink CLI in 1.5.0

As a result the REST API (i.e. the web server) must be running in order to submit jobs. On 19.06.2018 14:12, Sampath Bhat wrote: Hello I'm using Flink 1.5.0 version and Flink CLI to submit jar to flink cluster. In flink 1.4.2 only job manager rpc address and job manager rpc

Breakage in Flink CLI in 1.5.0

Hello I'm using Flink 1.5.0 version and Flink CLI to submit jar to flink cluster. In flink 1.4.2 only job manager rpc address and job manager rpc port were sufficient for flink client to connect to job manager and submit the job. But in flink 1.5.0 the flink client additionally require

Re: Flink CLI cannot submit job to Flink on Mesos

our flink app: > https://ci.apache.org/projects/flink/flink-docs- > release-1.3/setup/mesos.html#high-availability > > -* Flink CLI 1.3.0 on my local machine* (make sure you *use the same > version*!!) > - Add same HA configuration values in your flink CLI

Re: Flink CLI cannot submit job to Flink on Mesos

-file-flink-confyaml - Add the Mesos HA configuration values in your flink app: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/mesos.html#high-availability - Flink CLI 1.3.0 on my local machine (make sure you use the same version!!) - Add same HA configuration values in your

Re: Flink CLI cannot submit job to Flink on Mesos

Hi Stephan, So, do you mean to remove the “-m” param from the flink CLI call? And on the other hand, that I should add the Zookeeper configuration in both sides, the remote flink and locally in the flink CLI config, right? Regards On 31 Jul 2017, at 22:21, Stephan Ewen mailto:se

Re: Flink CLI cannot submit job to Flink on Mesos

31, 2017 at 6:27 PM, Francisco Gonzalez Barea < francisco.gonza...@piksel.com> wrote: > Hi again, > > On the other hand, we are running the following flink CLI command: > > ./flink run -d -m ${jobmanager.rpc.address}:${jobmanager.rpc.port} > ${our-program-jar} ${our-prog

Re: Flink CLI cannot submit job to Flink on Mesos

Hi again, On the other hand, we are running the following flink CLI command: ./flink run -d -m ${jobmanager.rpc.address}:${jobmanager.rpc.port} ${our-program-jar} ${our-program-params} Maybe is the command what we are using wrongly? Thank you On 28 Jul 2017, at 11:07, Till Rohrmann

Re: Flink CLI cannot submit job to Flink on Mesos

motely using the Flink CLI command line tool. We have tried different configurations but in all of them we get errors from AKKA while trying to connect. I will try to summarise the configurations we´ve tried. - Flink 1.3.0 deployed within a docker container on a Mesos cluster (using Marathon)

Re: Flink CLI cannot submit job to Flink on Mesos

to submit a job remotely using the > Flink CLI command line tool. We have tried different configurations but in > all of them we get errors from AKKA while trying to connect. I will try to > summarise the configurations we´ve tried. > > - Flink 1.3.0 deployed within a docker conta

Flink CLI cannot submit job to Flink on Mesos

Hello, We´re having lot of issues while trying to submit a job remotely using the Flink CLI command line tool. We have tried different configurations but in all of them we get errors from AKKA while trying to connect. I will try to summarise the configurations we´ve tried. - Flink 1.3.0