First, Spark worker not have the ability to compute.In fact,executor is responsible for computation. Executor running tasks is distributed by driver. Each Task just read some section of data in normal, but the stage have only one partition. IF your operators not contains the operator that will pull middle result from each task, like collect or show，driver will not store any data. Each Executor not store the end result in memory by default, unless your operator contains the operator that cache data to memory, like cache or persist.
-- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
