Hey,
I've a question regarding submission job in detached mode. We're trying to
submit job via YarnClusterClient in detached mode and the first one goes
correctly, but when we submit second job we get exception:
Multiple environments cannot be created in detached mode
at
org.apache.flink.client.program.ContextEnvironmentFactory.createExecutionEnvironment(ContextEnvironmentFactory.java:67)
at
org.apache.flink.api.java.ExecutionEnvironment.getExecutionEnvironment(ExecutionEnvironment.java:1060)
What we are doing - we create instance of YarnClusterClient and submit
PackagedProgram through it to Flink container running on Yarn. Within our
jar used by PackagedProgram we get ExceutionEnvironment in a way:
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
By default this submission is made in attached mode but we faced problems
with our threads hanging after job submission, that's we we decide to try
detached approach.
But it seems that we cannot submit more than one job (in detached mode)
because of ContextEnvironmentFactory code:
if (isDetached && lastEnvCreated != null) {
throw new InvalidProgramException("Multiple enviornments cannot be created
in detached mode");
}
Do you have any clue why submitting more than just one job in detached mode
is denied, or how we could then submit our jobs in different way?
Regards,
Piotr
