[
https://issues.apache.org/jira/browse/SPARK-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100236#comment-16100236
]
jin xing edited comment on SPARK-21530 at 7/26/17 12:56 AM:
------------------------------------------------------------
I will send follow-up PR soon.
Thanks [~tgraves]
https://github.com/apache/spark/pull/18735
was (Author: [email protected]):
I will send follow-up PR soon.
Thanks [~tgraves]
> Update description of spark.shuffle.maxChunksBeingTransferred
> -------------------------------------------------------------
>
> Key: SPARK-21530
> URL: https://issues.apache.org/jira/browse/SPARK-21530
> Project: Spark
> Issue Type: Improvement
> Components: Shuffle
> Affects Versions: 2.3.0
> Reporter: Thomas Graves
>
> We should expand the description of spark.shuffle.maxChunksBeingTransferred
> to include what happens when the max is hit. In this case it actually closes
> incoming connections so if the retry logic on the reducer doesn't happen
> could end up in failure.
> The way its currently worded doesn't tell users bad things can happen when
> the max is hit.
> introduced with https://github.com/apache/spark/pull/18388/files
> SPARK-21175
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
