Thomas Graves created SPARK-20589:
-------------------------------------
Summary: Allow limiting task concurrency per stage
Key: SPARK-20589
URL: https://issues.apache.org/jira/browse/SPARK-20589
Project: Spark
Issue Type: Improvement
Components: Scheduler
Affects Versions: 2.1.0
Reporter: Thomas Graves
It would be nice to have the ability to limit the number of concurrent tasks
per stage. This is useful when your spark job might be accessing another
service and you don't want to DOS that service. For instance Spark writing to
hbase or Spark doing http puts on a service. Many times you want to do this
without limiting the number of partitions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
