Hi, all fetch wait time: * Time the task spent waiting for remote shuffle blocks. This only includes the time * blocking on shuffle input data. For instance if block B is being fetched while the task is * still not finished processing block A, it is not considered to be blocking on block B.
according to the definition of fetch wait time, spark can process block A and fetch block B simultaneously, means communication and computation can be overlapped? is it applied to all the situations? especially in some cases the worker needs both A and B to start the work. thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/can-communication-and-computation-be-overlapped-in-spark-tp6348.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
