[
https://issues.apache.org/jira/browse/SPARK-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839788#comment-15839788
]
Hyukjin Kwon commented on SPARK-5786:
-------------------------------------
It seems they are documented, at least, in API docs, e.g.,
https://github.com/apache/spark/blob/4cb49412d1d7d10ffcc738475928c7de2bc59fd4/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala#L55-L61
> Documentation of Narrow Dependencies
> ------------------------------------
>
> Key: SPARK-5786
> URL: https://issues.apache.org/jira/browse/SPARK-5786
> Project: Spark
> Issue Type: Improvement
> Components: Documentation
> Reporter: Imran Rashid
>
> Narrow dependencies can really improve job performance by skipping shuffles
> entirely. However aside from being mentioned in some early papers and during
> some meetups, they aren't explained (or even mentioned) in the docs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
