I agree with your take regarding superficial stream environment distinction
and the difficulties that introduces for users.
To fix the immediate issue in Beam, it was necessary to duplicate
RemoteStreamEnvironment.executeRemotely
https://github.com/apache/beam/pull/7169/files#diff-6acb0479d563cfc
I'm only voicing my opinion here; these do not reflect in any way
long-term directions.
I wouldn't remove the execute() method; it's too important for a
convenient execution of jobs via the CLI/WebUI.
But I would like to get rid of this distinction of environments as their
existence implies
Thanks for taking a look.
Are you saying that the longer term direction is to get rid of the execute
method from StreamExecutionEnvironment and instead construct the cluster
client outside?
That would currently expose even more internals to the user. Considering
the current implementation in Remo
I'm not aware of any plans to expose this in the StreamExecutionEnvironment.
The issue would be that we would start mixing submission details with
the job definition, which results in redundancy and weird semantics,
e.g., which savepoint configuration takes priority if both a job and CLI
job s
Hi,
Currently it is not possible to submit a job with savepoint restore option
through the execution environment. I found that while attempting to add the
support to the Flink runner in Beam (
https://issues.apache.org/jira/browse/BEAM-5396)
I also found https://issues.apache.org/jira/browse/FLIN
