Re: DataStreamAPI and Stateful functions

2021-09-23 Thread Igal Shilman
No worries! Glad everything worked out! Cheers, Igal On Thu, Sep 23, 2021 at 2:42 PM Barry Higgins wrote: > Hi Igal, > Apologies you are correct. I had my wires crossed. I had been trying to > get everything working through my local ide before I deployed to our > ververica cluster. > I was only

Re: DataStreamAPI and Stateful functions

2021-09-23 Thread Barry Higgins
Hi Igal, Apologies you are correct. I had my wires crossed. I had been trying to get everything working through my local ide before I deployed to our ververica cluster. I was only able to get the code running through IntelliJ by following the steps below. Once I reverted the hack and changed the

Re: DataStreamAPI and Stateful functions

2021-09-23 Thread Igal Shilman
Hi Barry! Glad to hear that it works for you! I just didn't understand: a) what is "flink.yaml" perhaps you are referring to "flink-conf.yaml"? b) why is it bundled with the distribution jar? I couldn't find it there (nor it should be there) I've looked manually by: jar tf statefun-flink-distribut

Re: DataStreamAPI and Stateful functions

2021-09-23 Thread Barry Higgins
Hi Igal, I just wanted to give you an update on my deployment of stateful functions to an existing Flink cluster. The good news is that it works now when I submit my config with the statefun-flink-distribution. Thank you very much for your help. There was one gotcha and that was down to the requi

Re: DataStreamAPI and Stateful functions

2021-09-10 Thread Barry Higgins
Thanks Igal, I appreciate you coming back to me. I have quickly tried the fat jar solution as you've gone through it and am running into an exception: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Failed to execute job 'StatefulFunctions'.

Re: DataStreamAPI and Stateful functions

2021-09-10 Thread Igal Shilman
Hello Barry, I assume that by "we don't need another installation of Flink to manage the stateful functions." You mean that you already have a running Flink cluster and you would like to submit an additional Flink Job that executes a Stateful functions application? Then perhaps just try to submit

DataStreamAPI and Stateful functions

2021-09-09 Thread Barry Higgins
Hi, I'm looing at using the DataStream API from a Flink application against a remote python stateful function deployed on another machine. I would like to investigate how feasible it is to have all of the state management being handled from the calling side meaning that we don't need another in