Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-22 Thread Yuval Itzchakov
Hi Caizhi, This is our program printing out the status code, but it doesn't really matter. The point is that I have no ability to run a StatementSet through the WebSubmission Rest API without blocking ATM. On Wed, Dec 22, 2021 at 1:39 PM Caizhi Weng wrote: > Hi! > > This "Finished successfully w

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-22 Thread Caizhi Weng
Hi! This "Finished successfully with value: 0" seems quite suspicious. If you search through the code base no log is printing such information. Could you please check which component is printing this log and determine which process this exit code belongs to? Yuval Itzchakov 于2021年12月22日周三 15:48写

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Yuval Itzchakov
The job construction itself is a bit complex, but it can either be a StatementSet that's being filled, or there is some kind of conversion Table -> DataStream and then we put the transformations on the DataStream itself. Invocation looks like this: executionEffect = if (...)

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Caizhi Weng
Hi! You talked about status code so I guess you're speaking about the client that submits the job, not the job itself. Flink jobs does not have "exit codes", they only have status such as RUNNING and FINISHED. When you run your user code locally, it is running in a testing mini-cluster in JVM. So

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Yuval Itzchakov
I mean it finishes successful and exists with status code 0. Both when running locally and submitting to the cluster. On Wed, Dec 22, 2021, 08:36 Caizhi Weng wrote: > Hi! > > By "the streaming job stops" do you mean the job ends with CANCELED state > instead of FINISHED state? Which kind of job

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Caizhi Weng
Hi! By "the streaming job stops" do you mean the job ends with CANCELED state instead of FINISHED state? Which kind of job are you running? Is it a select job or an insert job? Insert jobs should run continuously once they're submitted. Could you share your user code if possible? Yuval Itzchakov

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Yuval Itzchakov
Hi Caizhi, If I don't block on statementset.execute, the job finishes immediately with exit code 0 and the streaming job stops, and that's not what I want. I somehow need to block. On Wed, Dec 22, 2021, 03:43 Caizhi Weng wrote: > Hi! > > You can poll the status of that job with REST API [1].

Re: Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Caizhi Weng
Hi! You can poll the status of that job with REST API [1]. You can tell that the job successfully finishes by the FINISHED state and that the job fails by the FAILED state. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-jobid Yuval Itzchakov 于2021年12月22日周三 02:3

Waiting on streaming job with StatementSet.execute() when using Web Submission

2021-12-21 Thread Yuval Itzchakov
Hi, Flink 1.14.2 Scala 2.12 I have a streaming job that executes and I want to infinitely wait for it's completion, or if an exception is thrown during initialization. When using *statementSet.execute().await()*, I get an error: Caused by: org.apache.flink.util.FlinkRuntimeException:* The Job Re