[
https://issues.apache.org/jira/browse/FLINK-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua Griffith updated FLINK-8137:
-----------------------------------
Description:
When submitting a new {{FlinkPlan}} using the {{StandaloneClusterClient}}, the
JobManager's REST API appears to hang until the job is submitted. The
submission time may be large enough to cause timeouts if the {{getStatistics}}
and {{createInputSplits}} methods of a job's {{RichInputFormat}}s perform
time-intensive tasks like running external queries. This is exacerbated when a
job contains many {{RichInputFormat}}s, since they appear to be initialized
sequentially. For a particular job with over 100 inputs, it's typical for the
API (and consequently the web UI) to be non-responsive for 45–60 seconds.
Would it make sense for tasks to have a {{Configuring}} state before the
{{Created}} state to provide greater visibility and indicate that the
JobManager is still healthy?
was:
When submitting a new FlinkPlan using the StandaloneClusterClient, the
JobManager's REST API appears to hang until the job is submitted. The
submission time may be large enough to cause timeouts if the getStatistics and
createInputSplits methods of a job's RichInputFormats perform time-intensive
tasks like running external queries. This is exacerbated when a job contains
many RichInputFormats, since they appear to be initialized sequentially. For a
particular job with over 100 inputs, it's typical for the API (and consequently
the web UI) to be non-responsive for 45–60 seconds.
Would it make sense for tasks to have a Configuring state before the Created
state to provide greater visibility and indicate that the JobManager is still
healthy?
> Flink JobManager API non-responsive during job submission
> ---------------------------------------------------------
>
> Key: FLINK-8137
> URL: https://issues.apache.org/jira/browse/FLINK-8137
> Project: Flink
> Issue Type: Bug
> Components: Client, Job-Submission, JobManager, REST, Webfrontend
> Affects Versions: 1.3.2
> Environment: Flink 1.3.2 running a batch job in Kubernetes.
> Reporter: Joshua Griffith
> Priority: Minor
>
> When submitting a new {{FlinkPlan}} using the {{StandaloneClusterClient}},
> the JobManager's REST API appears to hang until the job is submitted. The
> submission time may be large enough to cause timeouts if the
> {{getStatistics}} and {{createInputSplits}} methods of a job's
> {{RichInputFormat}}s perform time-intensive tasks like running external
> queries. This is exacerbated when a job contains many {{RichInputFormat}}s,
> since they appear to be initialized sequentially. For a particular job with
> over 100 inputs, it's typical for the API (and consequently the web UI) to be
> non-responsive for 45–60 seconds.
> Would it make sense for tasks to have a {{Configuring}} state before the
> {{Created}} state to provide greater visibility and indicate that the
> JobManager is still healthy?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
