Shixiong Zhu created SPARK-19182:
------------------------------------
Summary: Optimize the lock in StreamingJobProgressListener to not
block when generating Streaming jobs
Key: SPARK-19182
URL: https://issues.apache.org/jira/browse/SPARK-19182
Project: Spark
Issue Type: Improvement
Components: DStreams
Reporter: Shixiong Zhu
Priority: Minor
When DStreamGraph is generating a job, it will hold a lock and block other
APIs. Because StreamingJobProgressListener needs to call DStreamGraph's methods
to access some information, the UI may hang if generating a job is very slow
(e.g., talking to the slow Kafka cluster to fetch metadata).
It's better to optimize the locks in DStreamGraph and
StreamingJobProgressListener to make the UI not block by job generation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
